Arroyo is an open source distributed stream processing engine written in Rust. It is designed to efficiently perform stateful computations on streams of data. Arroyo lets you ask complex questions of high-volume real-time data with sub-second results.
The Arroyo project was started by a team of engineers from YC W23. They are passionate about making real-time data processing more accessible and affordable. They believe that Arroyo can help organizations of all sizes to take advantage of the power of real-time data.
The Arroyo project is still under development, but it has already been used by a number of organizations, including Plaid, Affirm, and Stitch Fix. The project is open source, so anyone can contribute to its development.
Here are some of the features of the Arroyo project:
SQL and Rust pipelines Scales up to millions of events per second Stateful operations like windows and joins State checkpointing for fault-tolerance and recovery of pipelines Timely stream processing via the Dataflow model Arroyo can be self-hosted, or used via the Arroyo Cloud service managed by Arroyo Systems. If you are looking for a powerful and efficient stream processing engine, then Arroyo is a good option to consider. It is still under development, but it has a lot of potential.