kibot

Kibot Updater

Kibot Updater is the free Windows tool that takes the self-extracting update archives published to Kibot's FTP server and merges them into an existing on-disk dataset. It appends new bars to daily and intraday files, replaces back-adjusted files in full whenever a split or dividend forces a rewrite, and exposes a command-line mode for scripted pipelines. It pairs with the FTP delivery channel described in Data updates; Kibot Agent covers the same job for customers who would rather not touch FTP at all.

What it does

Every Kibot update is packaged as a self-extracting .exe archive on the FTP server. The archive contains a Daily folder, an Intraday folder, an adjusted_files.txt manifest listing any symbols that were back-adjusted for a corporate action, and the KibotUpdater.exe binary itself. Running the archive launches the Updater, which reads the manifest and the two data folders and writes changes into the customer's local archive in place.

Two merge strategies run in parallel inside one update. For files that need a full back-adjustment, such as a stock that split or paid a dividend during the update window, the archive carries the complete new file and the Updater replaces the local copy outright. For everything else, the Updater appends the new bars to the end of the existing file. Splitting the work this way keeps update archives small for typical day-to-day refreshes and avoids re-downloading the entire history every time a corporate action lands.

Update archives are named with explicit start and end dates so they sort and select cleanly from scripts. The format is yyyymmdd.exe for daily updates and yyyymmdd-yyyymmdd.exe for weekly, monthly, and full archives, for example 20110404.exe for the 4 April 2011 daily update and 20110301-20110331.exe for the March 2011 monthly archive.

GUI mode

The simplest invocation is to double-click the downloaded .exe. The Updater opens a dialog asking for the destination folders, defaults to the paths set in config.xml from previous runs, and shows a progress bar while it unpacks and merges. The dialog reports per-file results in a log pane so an operator can spot a back-adjustment or an "access denied" failure as it happens.

Splits and dividends are handled silently. Whenever a file needs a full adjustment, the complete corrected version is shipped inside the archive and the Updater replaces the local file rather than trying to compute the adjustment locally. Kibot uses the same adjustment method as Yahoo Finance's historical-daily series, which makes a back-to-back comparison against Yahoo's UI a fair sanity check after a run.

Custom merge without the tool

The .exe archive is a standard self-extracting RAR. Renaming it from .exe to .rar turns it into a plain archive that 7-Zip, WinRAR, or any other RAR-aware tool can extract, which is the path of choice for customers running their own ingestion pipeline against a database rather than against the on-disk text files. Inside the archive are the same Daily and Intraday folders, plus the adjusted_files.txt manifest naming the symbols that have been back-adjusted. A custom merger needs only two rules: replace any file listed in adjusted_files.txt in its entirety, and append the remainder to the existing local file.

Command-line mode

The Updater accepts the same parameters from the command line that the GUI sets through its dialog. Driving it from a script lets a customer chain it behind a scheduled FTP pull and remove the operator from the loop entirely.

The general form is:

<executable> -d"<dest>" -<silent> -sp"<dest>|<intraday>|<daily>|<delete>|<regularsession>|<visible>"

The parameters break down as follows:

  • <executable>, full path to the downloaded update .exe.
  • -d"<dest>", destination folder where the archive is unpacked before merging.
  • <silent>, s1 hides the start dialog and the extraction-progress window, s2 hides only the start dialog. Omit the switch to keep the interactive dialog.
  • <intraday>, path to the existing intraday data folder that should be updated.
  • <daily>, path to the existing daily data folder that should be updated.
  • <delete>, 1 deletes the extracted files after the merge, 0 leaves them in place for inspection.
  • <regularsession>, 1 filters out pre-market and after-hours bars before merging, 0 keeps the full session.
  • <visible>, 1 shows the main UI during the merge, 0 hides it. Any errors are appended to log.txt inside the destination folder when the UI is hidden.

Example pipeline

A worked example for the 4 April 2011 daily update, applying it on top of a local archive at C:\Kibot\Data\:

c:\Kibot\20110404.exe -d"c:\Kibot\Updates\" -s2 -sp"c:\Kibot\Updates\|c:\Kibot\Data\Intraday\|c:\Kibot\Data\Daily\|1|0|1"

This unpacks the archive into c:\Kibot\Updates\, hides the start dialog while keeping the progress bar visible (-s2), merges the intraday and daily folders into the local archive, deletes the extracted files when done (1), keeps the full trading session (0), and leaves the UI visible during the run (1).

Calling KibotUpdater.exe directly

An alternative is to extract the archive yourself (or rename it to .rar and unpack it manually) and then run the KibotUpdater.exe shipped inside it against the unpacked folders. The parameters are the same minus the leading update-archive path:

KibotUpdater.exe "<dest>|<intraday>|<daily>|<delete>|<regularsession>|<visible>"

This is useful when the FTP pull and the unpack step already live in a separate stage of a pipeline, or when the same Updater binary is being reused across many archives.

Operational practice

Always run as administrator. The single most common Updater failure mode in support correspondence is a permissions error: a non-elevated process cannot write into the destination folder, cannot modify files that another program has open, and cannot register the cleanup step. Kibot support's standard guidance is to right-click KibotUpdater.exe, open Properties, and tick "Run this program as an administrator" so every invocation elevates by default (per support correspondence). One customer reported the issue plainly: "running as administrator solved my problem" (per support).

Watch the log when running silently. The visible=0 flag suppresses every dialog but still writes diagnostics to log.txt in the destination folder. Tail or scrape that file after each run instead of trusting the exit code alone; a missed back-adjustment or a locked source file shows up there but not in the process status.

Schedule for the calm window. Daily updates land on the FTP server before 8 AM Eastern (see Data updates, FTP server). Tuesday morning is the safest window for weekly archives and full archives that finish assembling over the weekend. Scheduling the Updater to run between 02:00 and 06:00 Eastern catches both daily and tick refreshes against a lightly loaded server.

Large archives split into chunks. Initial full-history archives can be tens of gigabytes for the bigger packages. Customers on home connections sometimes see the download stall after several GB; Kibot support routinely splits such archives into roughly 2 GB pieces on request so each piece has a realistic chance of completing, one customer's report described losing the download at 5 GB and again at 8 GB on a 14 GB archive before asking for splits (per support correspondence). The Updater itself does not need the splits to be rejoined: any RAR-aware extractor handles the multi-part archive transparently before the merge runs.

Symbol-removal mid-window. When a symbol is removed from the catalogue between two update points, the merge does not delete the existing local file. Operators that want a strict mirror should diff adjusted_files.txt plus the folder listings against the previous run rather than trust the on-disk state alone.

When to use the Updater versus Agent versus the API

The Updater is the right tool when the customer is already pulling Kibot's FTP archives, either by hand or with a scheduled FTP client, and wants the merge step automated. Kibot Agent is a step up: Agent handles both the download and the merge through one UI and one scheduled task, talks to the API directly rather than going through the FTP archive shape, and bundles its own resume and parallel-connection logic. The Kibot API is the right path when a pipeline is already written in code: it returns the same data the FTP archives carry, with finer control over date ranges and intervals, and no archive-unpack step at all.

Tick data is the one delivery-channel exception. The FTP server does not host tick files because of their size, so the Updater is not part of the tick refresh story; tick updates come from the API, from the per-file direct download links on the account page (typically driven with wget), or through Kibot Agent.

Key Takeaways

  • The Updater merges Kibot's FTP self-extracting update archives into an existing on-disk dataset, appending new bars and replacing back-adjusted files in place.
  • The archive is a self-extracting RAR containing Daily and Intraday folders, an adjusted_files.txt manifest, and KibotUpdater.exe; renaming the file from .exe to .rar exposes the same contents to any RAR-aware archiver.
  • File names embed the update window: yyyymmdd.exe for daily, yyyymmdd-yyyymmdd.exe for weekly, monthly, and full archives, easy to filter by date in scripts.
  • Command-line mode takes a single pipe-delimited -sp argument carrying destination, intraday folder, daily folder, delete-after-flag, regular-session-only flag, and visibility flag; pair with -s1 or -s2 to suppress dialogs in unattended runs.
  • Always run the Updater elevated. Permissions errors on the destination folder are the most common support ticket, and "Run as administrator" via the executable's Properties dialog clears the majority of them.
  • Silent runs write diagnostics to log.txt inside the destination folder; the exit code is not a complete signal on its own.
  • Splits and dividends are handled by full-file replacement using Yahoo Finance's adjustment method, so the merge is fast and Kibot output reconciles directly against Yahoo's historical daily series.
  • Tick data is not part of the FTP archive set and is therefore outside the Updater's scope; refresh tick history through the API, direct download links, or Kibot Agent.
  • Data updates, the four delivery channels in context, including FTP archive timing and update cadences.
  • Kibot Agent, bundles download and merge into one GUI plus scheduler, for customers who would rather skip the FTP layer entirely.
  • wget install, how to install wget on Windows, Linux, and macOS for the direct-download-link path that complements the Updater for tick data and ad-hoc pulls.
  • Kibot API, the programmatic alternative to the FTP-plus-Updater path.
  • Data format reference, the on-disk file format the Updater reads and writes.
  • Fastspring payment, how update subscriptions renew on a 30-day cadence.