OUR TECHNOLOGY
Windjammer accelerates spark queries by 3x while reducing infrastructure cost to one quarter. We offer seamless integration with AWS-EMR and GCP-DataProc and 100% SPARK compatibility. Install Windjammer without interrupting your workflow. We're eager to benchmark our engine against any of our competitor's, apples-to-apples. Our devoted team of engineers make it easy to get answers to all your technical questions immediately.
.png)
Windjammer Accelerator provides industry leadership in Cloudstore Analytics and ML
Why Spark Acceleration?
-
Massive scale of use of Spark
-
Spark’s JVM is very CPU intensive causing server sprawl, performance instability & management challenges
-
Spark does not fully exploit high bandwidth of today’s cloud storage systems, causing high query run times
-
Standard Spark fault tolerance requires persisting data at shuffle boundaries, motivating complex and expensive shuffle services
Windjammer Spark Accelerator
-
More efficient use of expensive CPU resources:
-
Native execution
-
MPP (massively parallel processing)
-
Dataflow clustered architecture eliminating JVM bottlenecks
-
-
Fully exploits S3/GCS cloud storage bandwidth
-
Aggressive parallel, asynchronous prefetch of analytics data sets
-
Eliminates need for complex shuffle service:
-
Checkpoint-based fault-tolerance uses reliable, high bandwidth S3 cloud storage: no need for special shuffle service while providing full query fault tolerance including spot instance and cluster interruptions
-
-
Transparent, 100% compatible
Windjammer cuts cluster size to a quarter while accelerating performance
Windjammer cuts customer costs to a third for Spark cloud deployments
.png)
Windjammer’s slashes CPU to a quarter or more
Windjammer boosts spark cloud store utilization by 3x
If you are interested in...
-
3x-5x increase in query throughput/vCPU to eliminate server sprawl
-
Making full use of S3/GCS cloud storage bandwidth to speed up your Spark jobs
-
Eliminating the cost and hassle of a shuffle service
-
Doing all this with a simple bootstrap extension in EMR cluster creation
-
Then you will be interested in deploying EMR and Dataproc with Windjammer’s Spark Accelerator
-
We look forward to working with you!
Patents
U.S. Patent No. 11,327,966 B1:
U.S. Patent No. 62/898,331, filed Sept. 10, 2019, Massively Parallel Processing with Precise Parallel Prefetching on Data Lake Cloud Object Stores
Published Patent Application No. 2022/0277006
.png){kind=logo}
