Historical Options Tick Data™ (HOT Data™) is an offering from Hanweck Associates that provides daily files that include trade and quote data for listed equities, index and ETF options that are reported to the Options Pricing Reporting Authority (OPRA). Historical data is available from June 2005.
These comprehensive, high quality, historical daily files can help improve trading efficiency by enabling sophisticated investment professionals to:
Currently, there are two historical data offerings available:
Both offerings are delivered as compressed files to reduce delivery bottlenecks and storage requirements. Only three days of data are maintained on the FTP servers and remaining history will be delivered on portable hard drives. The average size of the daily file for 2012 is approximately 57GB compressed or 115GB uncompressed. The complete database is in excess of 75TB compressed or upwards of 177TB uncompressed. Subscribers need to run WinZip (version 9.0 or higher) or have the 64-bit internal file pointers for LINUX/UNIX to uncompress the files.
Please note: Foreign Currency Option (FCO) data is not included.
OPRA consolidates market data from all participating US options exchanges and transmits the data across 48 separate transmission lines. From April 2006 to May 2, 2011 OPRA transmitted data across 24 separate lines; prior to April 2006, OPRA only transmitted data across eight separate lines.
Prior to the roll out of the Option Symbology Initiative (OSI rollout began in March 2010) the data was distributed in alphabetical order according to the first few letters of the option root symbol for each option series (not the underlying security). For example, the data on Line 1 previously contained option series with option root codes A-APV, Line 2 contained APW-BK, and so on. After the OSI rollout the data was distributed in alphabetical order according to the underlying symbol for each option series (not the option root symbol). Refer to page 90 in the OPRA Data Recipient Interface Specification available at http://www.opradata.com/specs/data_recipient_interface.pdf.
For each trading day, ISE Holdings captures the raw OPRA data feed and stores it in 48 individual files (one file for each line). In addition, the ISE Holdings provides a separate MD5 hash file that corresponds to each of the 48 files to allow for file consistency checking. Therefore there are 96 files per day beginning May 2, 2011.
Beginning in August 2006, the ISE Holdings also provides a daily file containing the option root codes for each underlying symbol traded at ISE. Therefore, there were 24 OPRA files, 24 MD5 hash files plus the underlying root code file for a total of 49 files for each day between August 2006 and May 2011.
The daily files are compressed using WinZip so users will need WinZip v9.0 or higher to unzip the files. Because the data files may each be larger than 2GB, any Unzip program (e.g., running on UNIX or LINUX) will need to have 64-bit internal file pointers to successfully decompress the files.
OPRA data consists of messages (quotes, trades, open interest, etc.) made up of ASCII characters that must be parsed in order to work with the data. The format of each of the OPRA message types is given in the OPRA Data Recipient Interface Specification (http://www.opradata.com/specs/data_recipient_interface.pdf).
Starting in April 2008, OPRA began encoding the messages using “FAST for OPRA”. FAST (FIX Adapted for STreaming) is an encoding algorithm that compresses the OPRA messages into FAST packets. This reduces the size of the OPRA ASCII messages by approximately 60-70%. Technical details on the FAST for OPRA specification for the latest changes can be found in the document: “FAST for OPRA: SIAC Technical Information for OPRA Data Recipients” available at http://www.opradata.com/specs/fast_for_oprav2_00.pdf.
The OPRA FAST packets must be decoded before processing the OPRA ASCII messages. Fortunately OPRA provides an off-the-shelf decoder.
The OPRA FAST decoder was designed to process multicast traffic and it will decode the OPRA FAST packets into the native OPRA ASCII messages. ISE HOT Data is available as flat files and since the decoder is designed to read multicast traffic, there are some small edits to the decoder are required to correctly decode the HOT Data files.
In November 2008 OPRA made some enhancements that required some changes to certain fields within the messages. The most noticeable change is that OPRA began to use millisecond timestamps. You will also need another document that has the specification for the Output Message Header that includes the expanded characters for ‘Time’ to support millisecond timestamps and for the expanded ‘Message Sequence Number.’ See page number 3 of the following document: http://www.opradata.com/specs/symbology_data_recip_20080708.pdf.
In the summer of 2005 an industry initiative began to develop a plan to eliminate the use of OPRA codes and come up with a standard to ensure that all option strike prices be represented in decimal format. The OSI committee included representatives from exchanges, vendors, broker dealers and the Options Clearing Corp. The Option Symbology Initiative plan was approved on December 5, 2006 and by the end of January 2007 the record layouts to be used throughout the testing and implementation phases of the project were approved. Detailed tests scripts were designed, approved and published in September 2008. A testing period was in progress from September 2009 through January 2010. There was a mandatory cut-over to the new data elements for the record layouts on February 12, 2010 and the roll-out of the new symbology, based on a fixed number of securities in each tranche, ran from March through May 2010.
The agreed-upon option symbology key has the following fields and represents the minimum data requirements used in the transmission of listed option contracts between exchanges, Options Clearing Corp and the participants. An example of the fields for the Apple 200 call option that expires August 21, 2010 is as follows:
|Symbol||Year||Month||Day||Call/Put (C/P)||Strike Price||Price Decimal|
In addition to decimal strike prices, another objective of the OSI was to create option symbols that would be more intuitive and less complicated for market participants to read. Let’s look at a couple of examples of the OPRA symbols before and after the project.
Here is an example of a legacy OPRA symbol pre-OSI:
Apple 200 Call, expiring 08/22/09 – APVHT
Here is an example of the new OPRA symbol post-OSI:
Apple 200 Call, expiring 08/21/10 - AAPL100821C00200000
Although the new symbol has many more characters, it is easier to identify which option series you are looking at. Keep in mind that this is just an example of the display and the actual display or order of the fields for the new OPRA symbol can vary depending on the subscriber’s implementation.
We strongly suggest you keep up with the latest OPRA notices at the following link: http://www.opradata.com/specs/data_recip.jsp.
The HOT Data end-of-day file is comprised of the captured raw OPRA transmission blocks and no cleansing or filtering is performed.
Please contact Geralyn Endo at 212-897-8171.
|Number of Months||Month Ending||number of days||compressed size (GB)||uncompressed size (gb)|
Support for ISE HOT Data is available from 8 am to 6 pm (ET) on market days. The contacts are:
|Business Issues||(212) 897-8171|
|Technical Support||(212) 897-0284|
No. In addition to the ISE and ISE Gemini data, we include the full OPRA daily file.
We collect the full OPRA broadcast from approximately 6:00 am to 5:55 pm (EST), and we include all quotes and trades from all exchanges as well as the OPRA flagged NBBO. The only data not included are foreign currency options (FCOs) that used to be part of the OPRA feed. Please refer to Section 7, Field Descriptions, for a complete list of available fields at www.opradata.com/specs/data_recipient_interface.pdf.
We currently we have tick history from June 1, 2005 to the present.
This flat file contains the tick capture of the native real-time OPRA feed and includes all trades and quotes from all participating OPRA exchanges, the OPRA BBO flags and other all other OPRA messages, including the open interest messages that are made available every morning. Detail of the content are provided in the following guide: http://www.opradata.com/specs/data_recipient_interface.pdf
We collect and deliver data using the standard OPRA format. The data is split up over a number of separate files in alphabetical order. Each file is compressed to reduce delivery bandwidth and storage requirements.
For all of 2010, the average daily file size is approximately 40GB compressed, or 75GB uncompressed. Because the data files may each be larger than 2GB, any unzip program for UNIX or LINUX will need to have 64-bit internal file pointers to successfully decompress the files. Windows users running WinZip will need version 9.0 or higher.
Through the end of 2010 the size of the data base going back to June 2005 is approximately 45TB compressed or over 100TB uncompressed. The break-down is as follows:
2005: 1.5TB compressed, 5.8TB uncompressed (June-December)
2006: 3.9TB compressed, 16.2TB uncompressed
2007: 5.9TB compressed, 24.2TB uncompressed
2008: 10.3TB compressed, 25.3TB uncompressed
2009: 8.5TB compressed, 15.8TB uncompressed
2010: 10.1TB compressed, 19.2TB uncompressed
All monthly files sizes through 2011 can be viewed on the "Monthly Files Sizes" tab.
There are three methods:
1. A subscriber will download up to five days of the daily OPRA tick history files from a FTP server
2. For all OPRA tick history files that are not available on the FTP server, the data will be delivered:
on a portable hard drive to subscriber for a separate one-time fee
over a cross connect for subscribers that have a direct connection to the ISE or are collocated at our primary data center located in Secaucus, NJ (Equinix)
Each new order request requires a signed agreement and order form that can be found on the “Fees/How to Order” tab. Subsequent orders only require an order form. Please email Geralyn Endo any paperwork or questions.
A subscriber will receive a user name and password within 24 hours of submitting the data agreement and order form.
The delivery time for ad-hoc requests depends on the amount of data and when we receive the order. If we receive an order before 10 am (ET), the following delivery times can be expected: 1-3 months of data – approximately 3 business days; 4-6 months of data – approximately 5 business days; 7-9 months of data – approximately 6 business days; 10 months or more - please contact us.
We deliver the data on external portable hard drives with an eSata connection, which are retained by the subscribers and can be reused.
Some OPRA feed handlers may have the capability to read this data. Alternatively, an OPRA parser can be written using the OPRA Data Recipient Interface Specification. You should also refer to steps on www.ise.com/hotdata under the 'Detail/Documentation' tab for the HOT Data User Guide.
In the current phase, we do not process requests for subsets of data based on symbols. However, OPRA now splits the tick data into 48 separate files, so each daily file consists of 48 separate zipped files. The split up is done based on the first letter of the symbol for each security. Therefore, File 1 = A, File 2 = B, etc., so if you were looking for all of the options starting with “Q,” you would not need to process the complete daily file. Please refer to page 89 of the OPRA Data Recipient Interface Specification. We are currently building a new product that will allow a subscriber to order just the historical data for specific symbols or other custom queries.
Yes. The terms specify that the data is for internal use only. We will consider redistribution requests on a case-by-case basis.
We capture the raw OPRA feed and do not impose any judgmental cleansing or filtering criteria of the data, which can impede the results of back testing.
No. However, there are a couple of third-party software providers that sell a data loader with an API. Please contact us for additional details.
For more information about, ISE HOT Data please contact our Market Data team.