Nothing is faster than memory access, don’t let too young programmers or IT company managers selling their stuff say that some database is faster with a blah blah blah algorithm. Right now you can buy a motherboard with 12 Gigabytes for a reasonable price. Load the quotes into a memory array, put it into a hash table, a c# Directory, a c++ Map and “Voila” you have the fastest database in the world. Oracle and Sybase should be dead at this time …
- Is there any reason for still having a database rather than playing with memory access?
- Hummm… Yes, I see only two reasons : the data formatting and the SQL querying!
When you receive data on a file, on an IP socket, on satellite dvb channel … you never have the same dump format. At this point you have plenty of “Commercial off-the-shelf”, ready-made products for database integration. And then you can use the SQL to do your well paid job.
It is a simple question of programming a pruning and formatting tool … so I’m proud to present “prune.exe” the “Swiss-Army-Knife” for huge data dumps. With this very little tool (programmed on the Kernighan & Richie way of life) I’m albe to adapt any file to the internal format my algo testing need…
Of course, in Algorithmic trading, you only need data for testing …
No need to be a visionary to understand that the Internet has changed the methods for trading the Forex. But what do we need to build from scratch a new and successful algorithmic trading software ?
You don’t need an expensive software architecture with powerful servers and very large bandwidth. No, you don’t need heavy architecture if you have the good algorithm (the one that nobody have and everybody want).
Of course if everybody was using the same algorithm, the problem will obviously be a question of power and speed. This is a validation of the expression : ideas are more powerful than the (brute) power.
The software architecture is not very easy to choose, Java is a good cross-platform development system, it can run everywhere and C# .net can also run either under windows, macos-x and linux with mono. Finally, the two choices are good, but you have to choose one. The element making a difference is the DotNet platform. Imagine you need to update a chart in real time for making a dashboard for Trading-Robot supervision. Integrating Reverse-Ajax with DotNet is really easy and it should be possible to code a robot directly in a website in C# without the needs of an expensive dedicated host.
We have choosen C# with the DotNet framework 3.5 and Ubuntu for the dedicated server used to log the quotes en real-time. For the database we choose MySql, simple, lightweight and also cross-platform.
Finally, the choice is not a problem if you know the different languages and frameworks, it is a question of feeling.
The real problem is inside the architecture … the code of the ATS itself!