Importing Data

From MultiCharts
Jump to: navigation, search

Importing .qmd data (MultiCharts data format)

  1. Open QuoteManager window
  2. In QuoteManager main menu select File, then point to Import Data and click Import Instruments…
  3. Select the file location and click
  4. Click Next
  5. Select the instruments to import
  6. Click Next
  7. Click Finish.

Importing ASCII data

The program supports importing from most known text (ASCII) files. The principal requirement for the file is a clear structure, data integrity and absence of logical errors.

See Invalid ASCII files section for information on the most common file faults.

The program is based on analyzing several hundreds of quote files, most in unique formats, which served as principles for file structure analysis. Despite successful automatic recognition of over 95% of files, there are still situations requiring the user to interfere, mostly because of improper file structure or if the program cannot make up a conclusion on its structure.

Note: For more information on known import limitations see ASCII Import Limitations section. Be careful when importing files.


How to import ASCII data

To import data:

  1. Open QuoteManager window
  2. Select a symbol (see Selecting Symbol(s) in Symbol list section). If the selected symbol contains quotes with the same timestamp as in the file, they'll be overwritten without warning. Make sure the symbol's price scale matches the price scale of the input data file. The program always uses the symbol's price scale when importing.
  3. Make sure the file you wish to import is a 1 Tick, Minutes or 1 Day. If a file contains bars, but has no static time increment, this is a Tick\Volume Bar file and must be processed using ASCII Mapping.
  4. On the File menu, click Import Data, or click the Import Data QM Import.png icon on the toolbar.
  5. Select ASCII or press Ctrl+I.
  6. Indicate the Path. The path can be typed in, pasted from the clipboard or selected using the mouse. The selected path persists after converting the file so on next operation the previous path will be used by default. If it is necessary to move up a level, click the Up icon. To change the method of displaying files click the rightmost icon Views and select an option from the drop down menu.
  7. Select the desired file from the indicated folder.
  8. If the Contents of the selected file field the information is garbled, choose the proper encoding in the File origin.
  9. Make sure the Resolution is valid. If the value is not correct, the possible causes may be the reason:
    • The Date field is not defined or not properly defined
      To set date:
      1. Click the Date column heading.
      2. Select Date and the corresponding template.
      3. Click Custom if there is no such template.
      4. Create a template. If the number of symbols in the date changes, use an asterisk * to replace the symbol, e.g. 9/9/2004 changed to 10/9/2004. The number of symbols changes from 6 to 7.
        • Allowed symbols in template for Date: m - month, d - day, y - year;
        • * sign means that length of chunk is undefined;
          Example: To convert date 1 Feb 2002 (2/1/2002) type *d mmm yyyy in the template field.
      5. Apply the created template by selecting it from the list.
    • The Time filed is not defined or not properly defined
      To set time:
      1. Click the Time column heading.
      2. Select Time and the corresponding template.
      3. If the template does not exist, click Custom.
      4. Create a template. If the number of symbols in the date changes, use an asterisk * to replace the symbol, e.g. 8:30 changes to 10:10. The number of symbols changes from 4 to 5.
        • Allowed symbols in template for Time: h - hour, m - minute, s - second,f - millisecond t - time modifier AM/PM.
        • * sign means that length of chunk is undefined;
        • @ sign splits the whole template and time modifier PM lexem.
          Example: To convert time 9:20 PM type *h:*m tt@pm in the template field.
          Note: Templates are stored in recognition engine's local settings. Time modifier can be in any language supported by current operating system.
      5. Apply the created template by selecting it from the list.
    • Wrong detection of the starting line of quotes
      To set Start at Row You should indicate how many lines to skip.
      After those parameters have been properly set, the program analyses the file again and sets the proper Resolution.
  10. Select Field Ask, Bid or Trade to define what kind of data the file contains.
  11. In the Time Zone please select the time zone of the quotes' origin.
  12. If the price or volume needs to be multiplied or divided, select a proper factor in Price Multiplier or Volume Multiplier.
  13. To select the time period for importing data set the Start Date/Start Date Time and End Date/End Date Time.


Invalid ASCII files

In most cases the invalid files are lacking the required fields, lacking data in some column, mixed up time and dates, irregular bar intervals. The program filters most faults and informs the user of them, but it is not possible in each and every case. Please see the analysis of typical invalid files below.


Invalid structure

Example 1. Two of the four necessary bar components are missing:
05-12-69  46.4  33.9
12-12-69  51.7  34.5
19-12-69  55.2  31

Example 2. Date is missing:
1.2061, 1.2065, 1.2061, 1.2065
1.2065, 1.2069, 1.2065, 1.2069
1.2069, 1.2070, 1.2066, 1.2066


Incomplete data

Example 1. File has a damaged string:
2/28/2002  10:45  31505  31530  31480  31480  125  86
2/28/2002  10:50  31480  31505  31480  31500  44    42
2/28/20

Example 2. File has variable number of columns:
СС12-D,10/01/1995,3.51,3.53,3.5,3.5,0,2643200,,,
СС12-D,11/01/1995,3.44,3.53,3.43,3.44,0,501600,0,,
СС12-D,12/01/1995,3.45,3.5,3.41,3.45,0,1545600,0.017544,,
СС12-D,13/01/1995,3.34,3.47,3.34,3.44,0,2353600,-0.038184,,
СС12-D 16/01/1995,3.39,3.41,3.34,3.35,0,1897600,-0.023325,,
СС12-D,17/01/1995,3.47,3.5,3.38,3.41,0,7425600,-0.011461,,
СС12-D,18/01/1995,3.45,3.5,3.41,3.5,0,766400,0.002903,-3.518377,


Logical errors

Example 1. The file has a 60 minute interval, but with 1759 instead of 1800:
<TICKER>,<PER>,<DTYYYYMMDD>,<TIME>,<OPEN>,<HIGH>,<LOW>,<CLOSE>
EURUSD,I,19980420,160000,1.1005,1.1010,1.1005,1.1010
EURUSD,I,19980420,170000,1.1003,1.1003,1.1003,1.1003
EURUSD,I,19980420,175900,1.1012,1.1012,1.1012,1.1012
EURUSD,I,19980420,200000,1.1012,1.1016,1.1009,1.1015
EURUSD,I,19980420,210000,1.1017,1.1020,1.1000,1.1006

Example 2. Time is not ordered chronologically:
GBPUSD,19890627,0,2200,15457,B,N,N
GBPUSD,19890627,0,2200,15462,A,N,N
GBPUSD,19890627,1,0100,15485,B,N,N
GBPUSD,19890627,1,0100,15493,A,N,N


ASCII Import Limitations

Important: Local data storage saves the following resolution types: 1 Tick, 1 Minute, 1 Day. All other resolutions are sampled from those.

Other data types may not be added to the database - use ASCII Mapping to process such files.