Giant end of year sale! Up to 50% off!
Home Buy Data Subscribe API Support My Account

Support

Any questions not included here? Please contact us.

Information presented here is organized into several sections. You can use the links below to skip scrolling and quickly access the section you are interested in:

  Ordering       Forex data
  Data format       Data quality
  Tick data       Adjusted and unadjusted historical data
  Futures data       Subscriptions, data updates and API access

Ordering and data delivery

I have placed my order - what happens next?

After you place your order and your payment is successfully processed, the maximum wait time never exceeds 24 hour period. In most cases, all data orders are available for delivery as an internet download immediately after we receive a confirmation of your order.

Please check your spam folder if you do not receive any e-mails from Kibot. If you email us or place an order and do not receive a response within 24 hours, please check your spam filter, junk mail folder, or whatever it is you use to filter out spam. There is a chance that the emails we sent to you may be accidentally identified and listed as spam.

If you do find our email in spam or junk folder, please make sure to mark it as "Not Spam or Junk" and to add kibot.com domain to your "Approved Sender" list to prevent this issue from reoccurring. To make sure that your e-mails to us are delivered, please use the online form to contact us. Do not send your first message to us from your e-mail client.

How secure is the order process?

Our order processing is 100% secure and it is powered by FastSpring a trusted global authority in electronic product distribution. All your private information is sent encrypted over the Internet via a secure connection. It is absolutely secure to purchase our products and data. For more information, please see this web page.

Flexible payment methods

We can accept credit and debit cards, PayPal, Amazon Payments, Bank Wire Transfers, Checks and certain Real Time Bank Transfers. Payments by all major credit cards are supported, including Visa, MasterCard, American Express, Discover, JCB, and UnionPay. Click here for more information.

How to get an invoice for a purchase order?

In order to generate an invoice for your purchase order please follow these steps:

1. Select one or more products on our web site and click on the "Continue", "Subscribe" or "Place Order" buttons. You will be redirected to a secure checkout page provided by FastSpring order processing service.
2. Click on the "Invoice" button on the right side of the checkout web page. If the "Securely save payment details..." check box on the right side is visible, make sure that it is NOT selected. This will activate the "Invoice" and "Wire Transfer" options.
3. In the "Payment Information" window that pops up, enter your contact information and your purchase order number.

You will then receive an invoice for the transaction before completing the payment by any standard method, including wire transfer.

You can test the process without entering any data. If any questions arise, please contact us.

Downloading historical market data

How are the files distributed?

When you buy any of our data packages, you will get an access to a secure section of the web site where you will find all your data and updates. Direct links are provided and files can be downloaded individually from our servers. You can also use our free Kibot Agent software to download and update data purchased from us or a third-party download manager. Updates are available from Kibot Agent, API or from the FTP server.

Can I use Python, wget or a download manager to download the files?

You will have access to direct download links on our web site that point to the most recent version of all files. You can click them and start the download in your web browser or download multiple files at once using other methods:

1. Login to the web site.

2. Click on the "Download..." link next to the data package that you would like to download.

3. You will be able to see the list of all instruments and their download links on the web page. There are plugins available for the most popular web browsers which can extract the links from a web page and start the download process.

4. To get a complete list of all links, click the "plain text format" link under "Getting download links" section on the page.

5. You can copy and paste the links into most download managers to initiate the download process. You can find a list of such programs here.

How to map download links to symbols or file names if I am using Python?

If you are using Python or any other programming language to automate downloads, you can get file names from the "Content-Disposition" HTTP header. For example:
Content-Disposition: attachment; filename=AAPL.txt

For more information, please use this web search.

How to use wget to download the files?

Login to the web site and open the download page for any of the data packages assigned to your account. Then, right-click the "plain text format" link select the "Save link as..." option in your web browser. Save the text file to your computer as, for example, links.txt. You can use the command line below to download the files and preserve the original file names:
wget --content-disposition --trust-server-names -i "links.txt"

How big are the files?

It depends on the data package you are interested in. For example, both daily and intraday data files for the "All Stocks and ETFs" historical data package compressed to a RAR or a ZIP archive are about 20 gigabytes in size. If you, for example, have a 20 Mbps connection to the internet, you can download everything in about 2 hours. You can calculate the approximate download time using this download time calculator.

Can I access the files from the Kibot FTP site?

Updates on our FTP server are in a compressed RAR archive format. Any popular archiving software like Winzip, WinRar or 7zip is compatible with our compressed files. Please note that FTP access is only used for updates. The source files on our servers are constantly updated and some of them are quite large which makes using the FTP protocol impractical.

Can I get a sample data file?

You can download complete one minute and tick data history for selected stocks and ETFs from the free historical data section on our Buy page. You can also download this zip archive with 1 minute unadjusted data for Wal-Mart during the regular trading session on 1/12/2010 and unadjusted data for OIH, Oil Services HOLDRs ETF on 12/31/2009.

Large amount of daily data for US stocks and ETFs is offered for free. You can easily download it and automatically keep it up-to-date using our Kibot Agent software.

Please contact us if you would like to have a small sample of some specific instrument's data. We will do our best to accommodate your request.

Where can I get the list of all symbols/instruments that you offer?

There are three separate lists with symbols: active instruments, active + delisted and delisted instruments only. TAB character is used as a separator to simplify parsing. The files are updated once per week. Columns are defined in the first line. Size and start date columns describe 1 minute files.


Historical data format

What is the format of your minute and daily data?

We use standard comma-delimited text files. This is the order of the fields in every 1 minute or higher interval file: Date,Time,Open,High,Low,Close,Volume.

Date values are in the "mm/dd/yyyy" format. Time values are using the ET (New York) time zone in the "hh:mm" format. Date and time values can easily be converted to any format of your choice.

We offer both adjusted and unadjusted data. Our intraday and daily adjusted data is back-adjusted for stock splits and dividends. Aggregated data records have a time value indicating the time when the bar opened. For example, a time stamp of 10:00 AM is for a period between 10:00:00 AM and 10:00:59 AM. All records with a time stamp between 9:30:00 AM and 3:59:59 PM represent the regular US stock market trading session.

For more information, please visit the Kibot data format for historical text files web page.

What is the format of your tick data?

The order of the fields in the tick files (with bid/ask prices) is: Date,Time,Price,Bid,Ask,Size. Our bid/ask prices are recorded whenever a trade occurs and they represent the "national best bid and offer" (NBBO) prices across multiple exchanges and ECNs.

For each trade, current best bid/ask values are recorded together with the transaction price and volume. Trade records are not aggregated and all transactions are included in their consecutive order.

The order of fields in our regular tick files (without bid/ask) is: Date,Time,Price,Size.

The order of fields in our 1, 5 or 10 second files is: Date,Time,Open,High,Low,Close,Volume. It is the same format used in our minute files.

What is the format of your aggregate bid/ask data?

The order of the fields in our aggregate bid/ask files is: Date,Time,BidOpen,BidHigh,BidLow,BidClose,AskOpen,AskHigh,AskLow,AskClose

The format is very similar to our standard one minute and higher interval files which are constructed by aggregating the execution price and volume. The difference here is that instead of using execution price we use "national best bid and offer" (NBBO) prices to get their open, high, low and close prices for any given time interval. Bids and asks in our files are from multiple markets and they represent the best prices or the highest bid and the lowest ask.

We record best bid/ask values whenever there is trade activity. Like with our regular tick data, current best bid/ask values are recorded for every trade.

A potential usage scenario may include using this data in your analysis to try to simulate the execution price for market orders and to estimate the potential slippage that may occur.

You can download sample aggregate bid/ask data for free from the free historical data section on our Buy web page.

Can I open the files in Excel?

Our default data format is already compatible with Excel. Instead of .txt, make sure that all files have the .csv extension. The resulting csv files will be properly interpreted in Excel and data will be shown inside separate columns.

You need to show the extensions in Windows first before renaming the files. Please see the "Renaming multiple files in bulk" section on this web page.

What time zone is used for US stocks and ETFs, Futures, Forex and other instrument types?

All our date/time values are created using the Eastern Time Zone (ET). If, for example, data originally comes from Chicago Board of Trade (CBOT) and the Central Time Zone, we will use ET (New York) time zone in our data in order to simplify data processing.

Time values are conformed to both EST (standard time) and EDT (daylight saving time). When observing standard time during autumn/winter, time values are 5 hours behind UTC time (UTC-5h). Similarly, when observing daylight saving time during spring/summer, values are 4 hours behind UTC/GMT time (UTC−4h). As a result, the stock market open which is at 9:30 AM ET is at 14:30 UTC during Eastern Standard Time (EST) (autumn/winter) and at 13:30 GMT during the Eastern Daylight Time (EDT) (spring/summer).

Our Kibot Agent software enables you to set a custom offset from -23 to +23 hours or convert date/time values to UTC automatically. You can also use the &timezone= parameter if you are downloading data using API.

If you need to convert our time values to your local time, here is the summary of when EST or EDT is used:
  - Prior to 2007 Daylight Saving Time (DST or EDT) begins on the first Sunday in April and ends on the last Sunday in October.
  - For year 2007 and after EDT begins on the second Sunday in March and ends on the first Sunday in November.
  - Time values are 4 hours behind UTC time (UTC-4h) for the spring/summer (EDT) period. Otherwise, they are 5 hours behind UTC time (EST).

You can get the exact dates for every year since 1918 when DST has started from this web site.

Are all market sessions included?

Our stocks and ETFs data includes pre-market (8:00-9:30 a.m. ET), regular (9:30 a.m.-4:00 p.m. ET.) and after market (4:00-6:30 p.m. ET) sessions. Trading for SPY (SPDR S&P 500 ETF) and some other liquid ETFs and stocks usually starts at 4 a.m and ends at 8 p.m. ET.

How far does your data go back?

We provide 62 years back in daily time frame and more than 26 years in minute-by-minute time frame. We also have over 15 years of tick-by-tick (including bid/ask) data. For intraday data, the earliest date on record is 1/2/1998 or the IPO (initial public offering) date for the company. There are also many delisted companies included in all of our data packages with the different start and end dates. You can find the list of all the companies with their start dates and file sizes on our Buy page.

Do I need to buy end-of-day (daily) data separately?

You do not need to pay for daily data if you buy any minute or tick interval data from us. All our tick and minute interval products have their end-of-day (daily) equivalent. Our daily data is included for free with intraday orders. For example, if you buy our S&P 500 historical 1 minute package, you will also get daily data free of charge. Similarly, if you buy tick data for the S&P 500 historical tick data package, we will include 1 minute and daily data for free with your order.

If one or more minutes are missing from your data does that mean that there were no trades?

There were no trades during that period of time. That does not mean the data is missing. This occurs more often during pre/after market sessions and with the less liquid stocks.

How can I reconstruct the missing bars and always get 390 bars per regular market session?

If you need 390 bars per market session, you can use the previous valid bar with volume to generate the missing bars. Here is an example:


Time,Open,High,Low,Close,Volume
------------------------------------------
15:00,10,10.5,9.5,10.4,100 <- has trades
15:01,10.4,10.4,10.4,10.4,0 <- missing bar
15:02,10.4,10.4,10.4,10.4,0 <- missing bar
15:03,10.4,10.4,10.4,10.4,0 <- missing bar
15:04,10.4,10.4,10.4,10.4,0 <- missing bar
15:05,10.4,10.6,10.2,10.5,200 <- has trades


Just use the last bar's close price for Open, High, Low and Close values and 0 for volume. If you later use this data to create technical indicators, data that is constructed in such a way is still correct and will not cause miscalculations. Also, make sure that in backtesting, you avoid placing orders during periods with no activity. Compare the hypothetical dollar amount on your entry and exit bars with the amount that was really traded.

Can I get different time intervals?

We offer several specific time intervals on our Buy page. Our tick data can be aggregated to any greater time interval. Similarly, one-minute files can be converted into 2, 3, 5, 10, 30, 60 minute files and imported into your software. Many software products can also internally convert one minute and tick data into higher time frames. We can also convert data for you to any interval greater than the one you originally purchased.

How can I convert your data to Metastock format?

You can use Metastock Downloader to convert our standard data files to any Metastock compatible format. For more information, please visit this web page. Our standard data format is explained here.

How can I convert your data to NinjaTrader format?

NinjaTrader minute, daily and tick format is natively supported by Kibot Agent software and Kibot API. If you are using API, just add "&ninjatrader=1" to your URLs to download data from our API in NinjaTrader format.

How can I load data directly into Microsoft SQL Server?

To store intraday data into SQL Server, use just one table for all the text files. You can use this table structure:

Symbol - char 6
Date - date
Time - time
Open - decimal 18, 4
High - decimal 18, 4
Low - decimal 18, 4
Close - decimal 18, 4
Volume - int

Create a clustered index on symbol, date and time columns.

To store daily data:
- remove the Time column
- use bigint instead of int for the Volume column
- create a clustered index on Symbol and Date columns

Then use the "Import and Export Data" wizard in SQL Server to import data from text files into tables you have created. To quickly retrieve data from the database, you can use a SQL query like the one below which returns all MSFT data from January 2017:

SELECT * FROM IntradayDataTable WHERE Symbol = 'MSFT' AND Date >= '1/1/2017' AND Date < '2/1/2017'

How can I automate Kibot Agent programmatically or using my own workflow?

You can execute the program using the command line and monitor the last line in its log file for the "successfully completed the task" or "task completed on" phrases. The alternative is to just wait for program to exit since all scheduled tasks are designed to close the program after execution. To see how to construct the command line, create a any scheduled task in the program and open Task Scheduler in Windows. You should be able to double click the created task and see the command line parameters under the "Actions" tab.


Historical tick data

Do you offer tick-by-tick data?

We have over 15 years of tick data including bid/ask prices at the time of transaction for all USA stocks, ETFs, Forex pairs and Futures contracts. Please visit our buy page for more details.

Does your tick data have bid and ask volume?

Only the transaction volume is recorded and best bid/ask prices at the moment of the transaction are taken from multiple exchanges. There are many price levels for bid/ask data coming from multiple exchanges and ECNs and each one has its own volume and the volume is constantly changing. We do not record bid and ask volume.

One of the popular high-frequency-trading strategies involves entering a large amount of orders on bid or ask side and then canceling them before the execution. It is almost an impossible task to save all those changes to our data files. According to US Securities and Exchange Commission (SEC), 96.8% of orders are cancelled. That makes any trading strategy that uses bid and ask volume unreliable.

Why are some prices in your tick data outside bid/ask spread?

Please note that, depending on the liquidity of the instrument, some small percentage of the trades may be outside the best bid/ask range.

The most common scenario is when the available liquidity is insufficient at the first bid/ask level. There are many price levels of limit orders on both bid and ask side. If a trade size is bigger than the first ask level for (for example) buy order, then the first price level on the ask side and levels above that price will be taken. The trade price then becomes the average of all price levels that were included in the trade. The average price will be higher than the best ask price at the moment of the trade.

Large block trades can be handled by the block desk at one of the major firms and then reported to the exchange. The reported trade price may very well occur outside the bid/ask spread.

Also the bids and asks in our files are usually across multiple markets and they represent the best prices or the highest bid and the lowest ask. Some trades may use liquidity pool that is unavailable to the public or simply does not represent the best bid/ask at the moment the trade is executed.

Trade may also be the result of the "marketable limit order" where a person can enter a limit buy order (bid) above the ask price (and vice versa) and execute the trade that way. If the order size is large enough, multiple price levels may be taken.

Please note that with Forex, there is no Last price, only Bid and Ask so, consequently, Bid may be higher than Ask and Ask lower than Bid.

Do you offer millisecond tick data?

Our Kibot Agent software supports downloading millisecond tick data for stocks, ETFs and Futures. The date/time format is 'hh:mm:ss.fff' where fff represents the number of milliseconds. Our standard tick data uses the 'hh:mm:ss' date/time format.

This data set starts from September 2014 and includes odd lots for stocks and ETFs and spread trade record for futures contracts. Updates are provided on a daily basis, but slightly later during the day.

If you purchase any of our tick data packages, you will have access to both regular tick data and millisecond data.

I am unable to to complete the download for large tick files over 20GB

This is most likely caused by a problem with your hard drive, especially if you use the compression feature in Windows. Windows limits the file size if the hard drive is fragmented and does not have enough free space.

Try opening "Administrative Tools/Event Viewer" and select "System" in the "Windows Logs" folder on the left side. You can use the "Filter Current Log" option on the right side of the window to isolate Critical, Warning and Error messages. If one or more hard drives are causing problems, you should see events with "disk", "NTFS" or something similar in the "Source" column.

Try to use the hard drive with a lot of free space and disable the compression for the destination folder. Right-click the folder and select Properties. In the General tab, select Advanced to open the Advanced Attributes. Under Compress or Encrypt attributes, uncheck Compress contents to save disk space.

The alternative solution is to split the download into two or more parts by using the appropriate start and end date values in the "Download Data" window in Kibot Agent.


Historical Futures data

How are your continuous contracts calculated?

The continuous contract rolls to the next contract as is. We do not back-adjust previous contract's data to current contract. Rollover is either on expiration date or 2-7 days before the expiration date. You can get the expiration date from the contract description on the historical futures data web page. You will see the expiration/delivery month and year in the "Description" column. Here is the futures tick data equivalent of the same list of futures contracts. Expiration is at 9:30 am ET (New York) time zone on the third Friday of the delivery month.

Is there a method for determining a settlement price from intraday prices?

The settlement price is set by determining the weighted average price over a certain period of trading, typically shortly before the close of the market. You can get the settlement price as received from the exchange in our daily data. The daily bar close value reflects the settlement.

Where can I get the list of exchanges used to compile historical Futures data?

You can get all the exchanges associated with individual futures contracts from this text file.

Why your daily (end-of-day) data for futures does not match intraday data?

Futures daily data does not represent aggregated intraday values. The close price in the daily data represents the "settlement price" and not the last price for the day. It is set by the exchange. Also, total volume in daily data includes "non-last-qualified trades" and volume for spreads which are counted by the exchange but not included in the intraday data. Non-Last-qualified trades are all trades that do not set the "Last" price. Daily data uses the 5PM to 5PM ET interval as a single trading session. So, for example, Monday trading session starts at 5PM on Sunday and ends at 5PM on Monday.

Why your daily or intraday data does not match other data sources?

Make sure that you are using proper symbols since our symbols for certain contracts may be different. For example, the symbol for our main wheat contract is W, while some other sources may use ZW. Click here to see the complete list of contracts that are available.

The other data source may be using price adjusted data (panama-canal shifted, ratio-shifted, weighted-average, calendar-weighted...). If that is the case, values from previous contracts need to be adjusted in order to match the current contract. The resulting values are not the actual transaction prices. They represent original values multiplied by some custom ratio.

Please note that our continuous data is unadjusted. We do not back-adjust prices and/or volume. Rollover for most of the contracts is on the expiration date. Our most recent data for the current contract should match their data but the further back in history you go, the more discrepancy you will see.

We offer continuous futures contracts and all individual contracts that are used to construct them. You can use any custom method you want for data analysis or to construct your own continuous contracts. This web page offers some suggestions and links to different papers and articles.

What time zone is used for Futures contracts?

We use the Eastern Time Zone (ET) for all our data. For example, even though Soybean Oil (BO) contract comes from Chicago Board of Trade (CBOT) and the Central Time Zone, we will still use ET (New York) time zone in our data in order to simplify data processing. You can get more information about the subject here.

How to import Futures data into NinjaTrader?

The program returns the "Instrument is not supported by repository" error message if a contract is not defined. You will need to manually add details about the missing contracts.
For more information, please visit this web page.


Historical Forex data

Why there is no volume in your Forex data?

Historical Forex data has no volume. It consists of bid and ask prices only. Unlike stocks, futures and many other markets, Forex market is not organized around a central exchange. As a result, transaction price and volume data is unavailable. Volume should be ignored completely in your analysis since it is not possible to get the total value of the transaction. If it is present in the file, it usually represents the number of times bid/ask prices have changed during any given time frame.

Do you use Bid or Ask prices to generate minute and daily data intervals?

Only Bid prices are used in our Forex 1 minute and higher interval historical data to generate Open, High, Low and Close values.

In order to preserve compatibility with our regular tick data format, we have included Price and Volume in the data. The order of fields is Date,Time,Price,Bid,Ask,Volume. Price is always equal to Bid and Volume is 1 or the number of transactions.


Indexes

Do you have a list showing all historical S&P 500 constituents?

Unfortunately we do not have a track record of all S&P 500 changes. Here is the best available free online source with history going back to 2000.

Are your stock index data packages survivorship-bias-free?

Some of the delisted and some bankrupt companies are included with every stock index package. We do not have a complete historical list of companies that were once part of the index nor do we have all the data for inactive stocks.

All active stocks are available. To find out what exactly is delivered once the order is placed, you can find all the files with their start dates and size together with the information about the included companies on our Buy page.

If you buy, for example, our S&P 500 historical data package, you will get over 800 symbols. The extra 300 companies are the ones that have been delisted.

In your stock index data packages do you include all historical data for a company or just the data since inception?

The entire available history for an instrument is always included, not just its data after the inception. Please refer to our Buy page to find out what exactly is included with every data package.


Data quality

How reliable is your data?

In order to see how reliable our data is, you can analyze complete tick and 1 minute files we offer as a free download. You can find the download links under the free historical data section on our Buy page. We encourage you to compare our data with any existing data you may have.

Will real-time data feed from my broker match Kibot data?

The feed you receive from your broker should have the same data as Kibot. Just like our data, the feed is combined and what you will get is the NBBO (National Best Bid and Offer) which refers to SEC rule that requires brokers to guarantee that customers receive the best bid/ask prices and include prices from all competing market centers (exchanges). It is unlikely that your broker will stream quotes from just one exchange if a certain instrument is trading on multiple exchanges.

Does your data include bad ticks (spikes)?

Our historical data is cross-checked, tested and verified for accuracy, but due to the lack of liquidity, spikes are quite possible in pre/after market sessions. When the data is processed on our servers, the filters are less restrictive for ticks outside regular session. You should take this into consideration and try to simulate orders in your backtesting only during the regular session between 9:30 AM and 4 PM. Also try to include only regular session when calculating indicators.

You can probably find a spike or two during the regular session. It is difficult to make a distinction whether or not those are real trades so the data is filtered against the most obvious errors only. You should be able to filter out the extreme prices in data if you compare them with the previous and/or next bars' prices.

I think I have found a spike in your data

Please report it to us but please note that many instruments are illiquid and therefore very volatile.

You need to calculate average true range for some period in the past. If the average daily range for an illiquid stock is, for example, 10% then any single 5% price movement should be considered normal. You usually need to ignore trades outside of the regular market session (from 9:30 AM to 4 PM) to reduce the number of false positives. If you find a potential spike, try to compare the prices to, for example, Yahoo Finance or some other third-party data provider.

Also please note that there were several flash crashes and outages on the exchanges in recent history. As a result, you can find a few prices that stand out even with highly liquid stocks like AAPL and GOOG.

Your daily data does not match Yahoo Finance historical prices

You are probably comparing our adjusted data with their unadjusted data. Our adjusted data set should match the last, "Adj Close" column. Our unadjusted data should be like the "Close" column on Yahoo Finance. Please note that Yahoo is adjusting prices only for splits and not dividends and we are using both parameters in our adjustments. So the only reliable way to compare the two data sets is to compare unadjusted prices. For more information, please visit this web page.

Why are some bars missing from the data?

There will not always be 390 bars per day during regular market session if you are working with less liquid instruments. Data is not recorded if there are no transactions during any specific time period.

Regular trading session is from 9:30 to 15:59. The last bar begins on 15:59:00 and ends with 15:59:59. Shortened trading sessions are from 9:30 to 12.59 which gives you 210 bars on such days. You can get more information about trading hours and holidays from NYSE.

Missing bars are rare in SPY, AAPL, MSFT or any other similar liquid instruments. In your backtesting, to take care of such gaps, avoid placing orders during periods with no activity. Always compare your hypothetical transaction volume on your entry and exit bars with the amount that was really traded on the exchange.

Also, a specialist on the NYSE exchange can delay the open for a particular stock. First bar during regular session  will then be after 9:30 AM which, again, produces less than 390 bars for that day. Such delays were more frequent in earlier years and before today's level of automation and computerization.

There is a way around this. You can generate all missing bars using prices from the previous bars and use zero value for volume in generated bars.

Why the close price from one bar is not the open price of the next bar?

This is not how the trades, prices and volume information is aggregated. For example, if you are using the 5 minute interval there is, theoretically, a 10 minute period between the trades when the prices on the bid or ask side can change dramatically. For example, there can only be two trades, one at exactly 9:30:00AM and the second one at 9:39:59AM. The first trade will be used to construct the 9:30AM bar and its closing price in the intraday data and the second trade will be used to construct the 9:35AM bar and the corresponding open price. The time difference between the close and the next open is almost 10 minutes and the two trades can have significantly different prices.

Why are some very liquid instruments have bars missing from the data?

You may discover that some very liquid stocks and ETFs like AAPL or QQQ have zero trades for long periods during trading hours. Trading on NASDAQ and NYSE was halted multiple times in the last few years. Click here to find just one example when NYSE trading was halted for almost four hours. This news article describes NASDAQ being closed for trading for 15 minutes in 2013.

Also, a lot of gaps may be incorrectly detected after 1PM ET. Shortened trading sessions are from 9:30 AM to 1:00 PM and some of the detected gaps may refer to after-hours trading which is far less liquid than regular trading session. You can get the list of all days with shortened trading hours since 1996 from Kibot API.

Why do I sometimes see a gap at the beginning of the regular trading session?

Sometimes, the specialist on NYSE may decide to start the trading session for a particular instrument a few minutes after the 9:30 AM official open time. This may occur even for very liquid securities and the missing bars in such case do not represent gaps or missing data. If having 390 bars per day for every single stock is crucial for your analysis, then you can simply reuse the previous bar's close and create the missing bar with no volume.

Why the closing prices in your daily files sometimes do not match up with what I see on Bloomberg/Yahoo and other data sources?

We use the first trade price as the Open price for the day and last trade price as the Close price in our files. There is a difference between the last trade price and the official closing price. The official closing price can be a transaction that occurred after the market close. In our daily data files the closing price for the day is the last price before 16:00 or the close price of the 15:59 bar in the intraday file.

For less liquid stocks, this can be before 15:59, but it is always the last intraday bar during the regular market session. If you use both intraday and daily data in your analysis, this increases the accuracy of your calculations since they will be synchronized.

The main idea behind our daily prices is to provide a way to calculate technical indicators for intraday trading. If you use both short and long term periods in the trading system rules, you will have more accurate signals if daily data matches intraday data.

If you use the "market-on-close" orders with your broker and you would like to simulate the execution price, you should probably use the official close price. Daily data with official closing prices is easily obtainable for free from third-party data providers. Click here to see an example for MSFT (Microsoft Corporation).

We have a separate page showing how incorrect Yahoo data can produce false results in your analysis.

Are "odd lots" or "non board lots" included in your data?

Our data does not include unfiltered "odd lot" transactions. The official volume from the exchange may differ slightly and include transactions smaller than 100 shares. Such quotes are marked as "non board lots" by the exchange. Non-board lot is anything below 100 shares or 1000 shares for penny stocks. Odd lots, by exchange definition, cannot set a last price which means that daily open, high, low and close prices cannot be set by odd lot transactions. Nasdaq and NYSE have included odd lot trading volume data in their daily volumes since October 2013. When present in intraday data, odd lots may prematurely trigger indicator based buy or sell signal or a stop, like, for example, ATR stop. For more information, please visit this web page.

Where does your data come from?

The price and volume data is taken from multiple exchanges and ECNs. For a stock traded at, for example, NYSE, the price is not from the NYSE exchange only. The recorded prices may come from other ECNs or exchanges, not just NYSE. Best bid, ask and transaction prices across multiple exchanges and ECNs are taken when constructing 1 minute and tick data. Feel free to analyze free historical data we offer on our Buy page. For more information about comparing our data to other sources, please visit this web page.


Adjusted and unadjusted historical data

Why use adjusted data?

A few dozen stocks and ETFs are adjusted for splits and dividends every working day. When historical data is adjusted, all the values are adjusted retroactively and multiplied by a calculated ratio.

You should use adjusted data in your analysis to create technical indicators and buy/sell signals which rely on them. If data is not adjusted for splits and dividends, indicators or any other calculated data series will produce large price gaps for the dates before and after the split or a dividend. As a result, your trading system may trigger a lot of false buy/sell signals.

If you need to get the price changes relative to the close of the previous day, you will get incorrect results with unadjusted data on dividend/split days. The difference can only be statistically correct if the data is adjusted.

Also, rather than using close/open prices which are more volatile, you can try to compare current day's data to previous day's average price. Try relaying on intraday and not the daily data. There is also an issue with the "official open/close price" which is often quite different than the intraday open at 9:30AM and close at 4PM. You can find more information about that here.

When should I use unadjusted data?

When you need to know the exact, real-life unadjusted price and volume on a certain date. All the values remain unchanged after corporate events like stock splits or dividends. Unadjusted data in almost all cases stays the same, but it may be periodically checked for errors and corrected.

What is your adjustment method for splits and dividends?

Since most of our customers are already familiar with Yahoo Finance historical daily data and are frequently comparing our data to theirs, we use the adjustment method explained on this page: http://help.yahoo.com/l/us/yahoo/finance/quotes/quote-12.html

Our adjusted data will in most cases match Yahoo historical prices, except for the daily closing price which in our case is the last transaction during the regular trading session. On Yahoo that may be the transaction after the close. To find out what the exact official close price was on a specific date, you can refer to the unadjusted "Close" column in Yahoo Finance data.

Is volume adjusted?

Volume is adjusted in both intraday and daily files. In unadjusted historical data it represents the actual number of traded shares. In both adjusted and unadjusted data, the (price * volume) formula always gives the exact dollar amount exchanged during a specific time period. For example, if a price is $10 and the volume is 100 shares, after a 2:1 split the price becomes $5 and volume 200 shares:

Price,Volume
$10,100
<--- 2:1 split
$5,200

Total dollar amount is in both cases $1000.

What is the difference between adjusted and unadjusted data?

There are two separate days in the example below. On the second day, the stock has a 2:1 split and subsequently the price reduces from $20 to $10. Here is how the unadjusted data would look like:

Unadjusted data:
$20
<--- 2:1 split
$10

The adjusted version of the same data corrects all the prices before the split:

Adjusted data:
$10
<--- 2:1 split
$10


Subscriptions, data updates and API access

How does it work?

There are several ways to download and keep your data up-to-date:

  • You can use our Kibot Agent software to download data and schedule automatic updates.
  • You can download updates from our FTP server and use your own custom method or the Kibot Updater tool to update your existing data.
  • You can use our historical data API to download the data from our server programmatically.
  • You can use direct download links to complete files that are available from the 'My Account' section of the web site. Plain text with all the links is available for use in third-party download programs.

How can I cancel a subscription?

Cancellation process is very simple and you are free to cancel an existing subscription whenever you want. All orders on our web site are handled by FastSpring. When you purchase any product with them, you are getting access to a protected area of their web site. From there, you can check the details of your subscription and cancel it directly.

If you have a subscription notification e-mail message, it should contain a link you can use to access the subscription; please follow the instructions in the message to edit or cancel the subscription. You can also contact us directly to cancel or make changes to your subscription. Please e-mail us at least 48 hours before the next billing date.

How can I subscribe?

  1. During ordering process. Select whether you want to subscribe to data updates while placing your order for any of our data packages.
  2. Using a direct link. You can use this web page or select subscription packages above to place your order for data updates any time after your initial purchase.

How to get access to Kibot API?

Access to Kibot API access is part of our "Premium" subscription. Guest access is also available and other subscription packages offer API access to daily or EOD (end-of-day) data. For more information, please visit this web page.

Should I wait until the beginning of the next month before subscribing?

No. Update services are renewed automatically on a monthly basis. Charges are made every 30 days starting with your initial order date - not when the calendar month changes. You can sign up towards the end of a calendar month and still get a full 30 days value.

Which payment types can I use for subscriptions?

Most of the payment methods currently available with FastSpring can be offered for subscriptions. Payment types include Paypal, credit and debit card, Amazon, wire transfer, check and cash payment.

What happens when the first rebilling is due?

If you have chosen to pay for the original order with a credit or debit card, FastSpring will automatically send an e-mail to remind you of the upcoming rebilling for this subscription. If the rebilling amount is successfully charged to the card, you will be able to continue using the product without interruption. If you have chosen to pay by wire transfer, check or cash, you will receive payment notification in time before payment is due. This invoice mail contains all relevant payment information.

What happens when the credit or debit card has been successfully charged (or when the payment has arrived)?

Following payment, you will receive your invoice and payment confirmation via e-mail and can continue to make use our subscription services.

What happens if the credit or debit card cannot be successfully charged or if no payment arrives?

Before the start of the next billing period you will receive a rebilling notification, informing that the unit price for the next subscription period will be charged to your card. If this automatic attempt to charge a credit/debit card fails, you receive an e-mail informing that charging the card was not possible or not effected and that the subscription has therefore been cancelled.

What happens if payment arrives later?

If payment arrives after the due date, you receive a mail, informing that the subscription has been reactivated and that you can therefore make use of the subscription product again.

Does the subscription resume with a new valid date?

Yes, the subscription is reactivated on the date that payment was made. This is the date used for calculating future rebilling.

Do I need to store my credit or debit card information?

You can choose not to store your payment information with FastSpring and pay manually. You will receive a rebilling notification when the next subscription period is due. We, however, recommend storing your data securely with FastSpring in order to automate the process and prevent any service interruptions.

How can I update my credit or debit card information for my subscription?

please find the last receipt e-mail you have received from FastSpring. It should contain a link to a secure page that allows you to update payment information on your subscription. This link is included in every receipt email which is sent to the customer, including the first time and all re-bills.

How can I update my existing files with your data updates?

We have made the process of updating your existing data files very simple. You can use Kibot Agent software to schedule and automate download tasks. If you download updates from the FTP server, just start the self-extracting archive and your existing files will be automatically updated by our Kibot Updater tool. You can automate the update process by using the built-in command line mode in Kibot Updater.

Is tick data included with the subscription?

By default, only minute and daily data intervals are included with the subscription as specified here. Tick data, more history or any other interval or data package not included with the subscription needs to be purchased separately. Please see our Buy web page for the complete list of data packages that we offer. Please note that all additional data purchases are one-time only events and that your basic subscription price remains the same regardless of the amount of data added to your account.

Is updating tick data supported?

Updating tick and any other interval data is fully supported. The only exception is our FTP server which does not store tick files and their updates because of the large file sizes. Tick data can be downloaded and updated using API, direct links to the file on the web site or with our Kibot Agent software.

How much does the data update service cost?

Data update service is billed on a monthly basis. With every data purchase, you are entitled to free quarterly data updates for one year. If you realize that you need more frequent updates, you can subscribe to the data update service at any later time. If you subscribe to the Premium subscription package, you will also have access to Kibot API and daily, weekly and monthly updates. Please visit this web page for pricing.

When are your FTP updates available?

All the updates are ready before the market opens at 9:30 AM ET (New York) time. Everything is usually ready before 8 AM. Tuesday morning is the best time to download weekly updates and full archives that are generated over the weekend. You can simplify the process by checking for new daily updates Tuesday - Saturday after 8 AM and for full and weekly archives on Tuesday after 8 AM. If you download full archives between Saturday and Tuesday they may be incomplete or corrupted.

Can I get a free access to Kibot API for testing purposes?

Access to daily data is free. You can use the guest account for testing. You can find documentation and ready-to-use examples here. We also offer free Kibot Agent software which uses our API internally. You can start and schedule downloads and updates from API with a few mouse clicks without any programming knowledge.

Why am i getting an error when trying to place an order?

All orders on our web site are handled by FastSpring, a third-party order processing service. They are using a risk analysis algorithm for all transactions. The transaction may be rejected, for example, if the country from which you are connecting differs from the country of the bank that issued the card. Avoid using a proxy server and try to use the internet connection from your provider without any obfuscations. Also, avoid using a free anonymous e-mail addresses like ProtonMail. E-mail accounts from Gmail, Yahoo, Hotmail and similar services should work.

Your funds may be "pre-authorized" in the process but that does not mean that the account was actually charged. A pre-authorization is a temporary hold on funds in a customer’s account that lasts typically around 5 days. Unlike an actual transaction, it simply puts an official hold on those funds until the post-authorization comes through. It can take anywhere from 5 to 10 business days for the funds to refunded which is out of our control as the bank is the one that has performed the pre-authorization.

Can I get a free access to Kibot API for testing purposes?

Access to daily data is free. You can use the guest account for testing. You can find documentation and ready-to-use examples here. We also offer free Kibot Agent software which uses our API internally. You can start and schedule downloads and updates from API with a few mouse clicks without any programming knowledge.

Here is how simple it is to get data from API:

To login you can use the guest account:
http://api.kibot.com/?action=login&user=guest&password=guest

To get last 10 days of adjusted data for MSFT (Microsoft):
http://api.kibot.com/?action=history&symbol=MSFT&interval=daily&period=10

Last 10 days of Unadjusted data for MSFT (Microsoft):
http://api.kibot.com/?action=history&symbol=MSFT&interval=daily&period=10&unadjusted=1

Kibot Agent process is not exiting when the task is finished

Make sure that Kibot Agent tasks in Task Scheduler have the "Run only when the user is logged on" selected. If this is not the case, the system will keep the Kibot Agent process running in the background.

If that does not help, you can open each task in the Task Manager and in the Settings tab enable the "Stop the task if it runs longer than..." option

Also, make sure that the program is always started "As administrator" so that it has full access to the scheduled tasks feature in Windows, destination folder and the files it needs to update. You can right-click the program's exe file or shortcut and select the Properties option. You will then be able to instruct Windows to always run the program "as administrator".

What are your data use terms and conditions?

You can find our license agreement here. We will try to accommodate any custom data usage scenarios. Please contact us with your requirements and we will get back to you as soon as possible.



Any questions not included here? Please contact us.