As part of his PhD research at CWI in Amsterdam, Azim Afroozeh researcher has developed two innovative tools that significantly speed up the processing of large datasets. These tools – FastLanes and ALP – are designed for analytical queries on column-based file formats and were recently released as open source. They aim to make data analytics more efficient across various industries.
The innovation centers on a redesign of how analytical data files are stored and accessed. FastLanes introduces a new columnar file format that performs more efficiently than existing standards like Parquet. ALP is a customized compiler that automatically generates optimized access code for these files. Used together, they deliver up to three times faster performance for common queries, without requiring more memory. The tools were developed as part of the NWO BrAIN programme, in collaboration with the University of Amsterdam and VU Amsterdam.
This technology is highly relevant for companies working with large-scale data, such as in AI model training, financial analytics, or medical imaging. Startups and R&D teams can reduce processing time and computational costs by adopting FastLanes and ALP. Because both tools integrate with existing infrastructures and are available under open source licenses, they are easy to adopt and adapt for custom use cases.
Read the full news release from CWI
Picture: Shutterstock (background) and CWI/Minnie Middelberg
Looking for partners to collaborate. Or looking for a certain expertise? Or would you like to locate your business in the Amsterdam Science Park? Drop us a line and we help you to find a perfect match.
Subscribe to our LinkedIn-newsletter and we will keep you updated on all that our park has to offer. It will arrive to your inbox five times a year and you can unsubscribe easily at any time.
Subscribe