Regular visitors will have noticed ever increasing amounts of spam on the Trading Gurus Community Forum. On November 24th 2012 we implemented some changes:

1. New users will be moderated 2. Registered users who have a post count of zero will not be able to view attachments or download code. 3. Registered users who have a post count of zero will not be able to create a new thread

As an additional countermeasure we anticipate introducing a modest paywall in the near future.

If this is your first visit here our apologies for the inconvenience.

Existing users may wish to contribute their ideas to this thread.

Results 1 to 8 of 8

Thread: Managing forex data

  1. #1
    Junior Member
    Join Date
    Jul 2010
    Posts
    19

    Managing forex data

    Here's my starter for 10.
    To do any form of data mining on forex data, we need need:

    1) Accurate historical data, ideally tick. There is of course Metatrader data from various companies, including Alpari. A source I have used in the past is Gain Capital

    2) A store for the data. We could of course just dump the data into Excel, and get surprisingly far. More realistically, to do any serious stuff, we need a SQL database. This is where things get interesting. We either hook up to the database through a programming language, OR we have to have powerful built-in procedural extensions, in which case it is perfectly feasible to manage mining data that way.

    I think for us amateurs, who don't want to fork out megabucks, options are:

    - MS Access: complete (if picky) SQL and good programming support (VBA), however, in my opinion Access is just too non-enterprise.

    - MySQL (open source): free, fast and no size limit. However, SQL limited outside of standard operations and no Analytics which is a real pain. Stored proc language is far too limited to do anything serious. Good choice if willing to hook up and program via your fav programming language.

    - SQL Server Express: comprehensive SQL + T-SQL procedural, powerful, but size limited.

    - Oracle Xpress: comprehensive SQL (with extensive Analytics) + PL/SQL procedural + extensive and very powerful built-in packages, but sadly, size limited.

    - Postgress (open source): I have never used this, but have heard that the DB is almost as stable as MySQL, and has a PL/SQL like programming language. Probably worth investigating.

    For what its worth, I use Oracle Xpress (because I love the Analytics + the built-in packages) or I go to MySQL if I hit size limits.

    What's the DB of choice for others doing forex data analysis?
    Last edited by pkotak; 08-31-2011 at 11:42 PM.
    Like a long-legged fly upon the stream, His mind moves upon silence. - W.B. Yates

  2. #2
    Administrator Jim's Avatar
    Join Date
    Apr 2010
    Location
    South West England
    Posts
    491
    Hi Prashant,

    Personally I use MySQL, mostly on a server somewhere in the clouds. I have an irrational fear of using software for which I do not possess the source code. Especially if it says Microsoft on the label!

    Mind you, I don't it use for any complex data analysis. Not yet at least!

    Cheers,

    Jim
    Reality is merely an illusion, albeit a very persistent one - Albert Einstein

  3. #3
    Junior Member
    Join Date
    Jul 2011
    Location
    Near Monaco at the French Cote d`Azur.
    Posts
    17
    Hello,

    let me suggest freeware: openoffice/libreoffice calc or, maybe better suited, gnumeric .

    If you want a database, I'd also venture for MySQL, as it's easily available/installed as part of a "LAMP" or "WAMP" package.

    I think that generally freeware is better suited as it's simpler to get more people involved in contributing/backtesting.

    Regards,
    pippa
    It takes years to become an overnight success

  4. #4
    Junior Member
    Join Date
    Jul 2010
    Posts
    19
    Hello pippa

    Both SQL Server Express and Oracle XE are also free, but size limited.
    There are ways round this (with Oracle at least), with multiple installs on a Linux VM, creating a powerful nest of distributed DBs on a single physical machine, but somewhat complex to setup, and in fact, program/use. The enterprise versions of both DBs are expensive, of course.
    MySQL's great, and I use it with perl or .NET, however there's no denying that the SQL on offer is primitive in comparision.
    Like a long-legged fly upon the stream, His mind moves upon silence. - W.B. Yates

  5. #5
    Junior Member
    Join Date
    Jul 2011
    Location
    Near Monaco at the French Cote d`Azur.
    Posts
    17

    Stanford University FREE classes on Artificial Intelligence & Intro. to Databases

    Well, thanks to your answer, I checked out the link. Interesting.

    For people with better understanding/higher capabilities than my 'copy & paste' trials,

    Stanford University offers free courses (via web/video) on
    Artifical Intelligence and
    Introduction to Databases.
    It takes years to become an overnight success

  6. #6
    Administrator Jim's Avatar
    Join Date
    Apr 2010
    Location
    South West England
    Posts
    491
    Hi Pippa,

    Quote Originally Posted by pippa View Post
    Stanford University offers free courses (via web/video) on Artificial Intelligence and Introduction to Databases.
    Thanks very much for that most interesting piece of news. It seems there's also a course on Machine Learning, which Ray is very interested in learning more about himself.

    Cheers,

    Jim
    Reality is merely an illusion, albeit a very persistent one - Albert Einstein

  7. #7
    Junior Member
    Join Date
    Jul 2012
    Posts
    1
    Feel comfortable while work with MySQL to manage Forex data !! i am not trying those software which i do not posses the source code and installing problems. Business Management Course Online

  8. #8
    Junior Member
    Join Date
    Jun 2012
    Location
    Ipswich, Suffolk, UK.
    Posts
    7
    Just a quick point - Postgres is much more than almost as stable as MySQL.

    If I needed to use open source I would choose Postgres over mysql everytime.. I would consider it over MsSQL in certain instances too.

    http://datachomp.com/archives/im-lea...-for-postgres/

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •