Leveraging the Power of Scala with Apache Flink
Have you ever felt like your Flink Applications needed something more? Maybe you have wanted to take a
Data Observability for Apache Flink® What the Flink Happened? Data Observability for Apache Flink® What the Flink Happened?
Monitor data applications smarter with Apache Flink® and Datorios. The first observability platform that uses metrics, logs, and tracing to diagnose real-time data, fix hidden issues, and save the day.
Watch Datorios In ActionWatch Datorios In Action
For streaming data developers seeking complete observabilityBecome A Data ExplorerFor streaming data developers seeking complete observabilityBecome A Data Explorer
Time Window Analysis
Visualize various time windows like tumbling time/count, sliding, or session windows. Identify the events that occur within a given window and detect missing events for a comprehensive view of your data.
Dynamic Event Visualization
Track events on a timeline graph, allowing for detailed analysis of each event’s journey during processing. Zoom in to specific timeframes for focused examination.
State Viewing
Monitor state evolution throughout operators event processing. Easily identify state changes occurring during processing.
Event Querying and Filtering
Utilize a versatile query language to extract specific operator states or event data, filter by values or keys, and target any processing period. Mark events for easy identification, such as flagging “late arrivals” within a defined timeframe.
Kafka to Flink Integration for the First-Time: Actionable Insights
In this follow-up article (see part 1), building on my initial explorations with Apache Flink, I aim to dive into
Apache Flink Deduplication: Key Strategies
Continuing on my Apache Flink Journey it’s time for some real world use cases. Now that I have
Security at DatoriosWe Are SOC 2 Type II CertifiedSecurity at DatoriosWe Are SOC 2 Type II Certified
FAQFAQ
Does Datorios have access to my data?
Datorios extracts certain client statistical information. All customer data remains visible only to the customer.
What data is collected by Datorios?
The Datorios client collects record metadata, state data, window details, along with Logs and Metrics produced by Flink.
How is data transferred to Datorios?
The data collected by the Datorios client is either hashed or encrypted based on your preference, and then it is uploaded to Datorios using SSL to our internal backend.
Can I delete my data at any time?
Yes, you can permanently delete any job and its associated data by navigating to the job screen and clicking the trashcan icon. To remove your organization, go to the organization screen and click the wheel cog at the top right corner.
What Flink languages does Datorios support?
Datorios supports Java, Python, and Scala.
What versions of Flink does Datorios support?
Datorios supports Flink version 1.6.x and above. For specific requirements, please contact us.
Can I run multiple jobs in parallel with Datorios?
Yes, Datorios allows you to run in either session mode or application mode.
What installation types are supported by Datorios?
Datorios supports installation via Docker and Kubernetes. For other installation types, please contact us.
Is Datorios operational at all times?
Datorios continuously collects metrics and logs, similar to other observability products like Datadog and Prometheus. Tracing data is collected only when triggered by specified metrics exceeding thresholds.
Can Datorios be installed in Kubernetes?
Yes.
Can Datorios be installed using Docker?
Yes.
Does storing large amounts of data make Datorios expensive?
No, Datorios uses a cache that continuously overwrites itself until a critical event is recognized. Only three minutes of data before a critical event are stored.
Is there a free version of Datorios available?
Yes, click here to get started.
Are there sample jobs available for testing in Datorios?
Yes, please refer to our GitHub and the preloaded jobs available in the application.