DataPoint endeavors to offer drag-and-drop import of any short-duration traffic count file. When we set this goal, we knew we had our work cut out for us. Surveying the most common software and formats was the easy part.
First, imported files must be classified. All file types (csv, pdf and worksheets alike) must be compared against known signatures and confidently identified as, for example, a Miovision 4-bin midblock csv with 24hr data in 15m intervals. It is equally important to prevent false-positives when novel formats are encountered for the first time.
Second, count data must be parsed. There are many existing solutions for extracting data from text files, pdfs and worksheets. However, these tools typically err on the side of returning something instead of nothing. Results are frequently incomplete or column-shifted. Extracting multiple tables or complex headers is problematic. The ability to permit certain structural nuances and prohibit others is limited, as is transparency into failures.
DataPoint uses an innovative file processing framework designed from the ground up to answer these challenges. It provides fine controls over classification, validation and extraction of any count file format. Users don’t need to know where files came from or what software was used, just drag it into the browser and let DataPoint handle the details.
Stay tuned for blog posts on how DataPoint validates your data, including customizable enforcement of tolerances set by your agency.