Apache Flink is an open-source, unified, stream-processing and batch-processing framework for stateful computation. Similar to other BigData products under the umbrella of Apache Foundation, Flink is not easy to get up and running in production without a proper automation. Likely for us, there are products for managing Flink, and there are companies offering Flink as managed service too. There are also few open-source products, which are a good option when the budget is an issue.

I have a preference for open-source software, and I would choose a solution which is integrated with Kubernetes and implements the Kubernetes Operator pattern. I like the idea that clusters and jobs are defined using custom resources which can be managed using the usual tools for Kubernetes. The custom resources can be saved in a git repository and deployed in the same way we usually do for other resources, which means we can use tools such as Helm or Flux.

We can do all of that using a custom controller, the operator, which takes care of executing the required tasks automatically. We need a custom controller because Flink is a stateful engine, and the standard controllers available in Kubernetes are not enough for managing Flink’s state.

I can’t vouch for other operators, but I can speak for the Kubernetes Operator I am developing. Flink K8s Toolbox is my solution for managing Flink on Kubernetes. The toolbox is a single command line tool which implements a Kubernetes operator, a Flink supervisor, and a CLI. The operator automates the process of managing clusters and jobs, and it simplifies the deployment of Flink on Kubernetes. The supervisor ensures that the Flink cluster has the expected state. The CLI interprets commands and submits requests to the operator.

I have created Flink K8s Toolbox to support the use cases I encountered in my job, but I hope it might be useful for other people too. If you have a use case which is not supported yet, please get in touch, you can help me to improve the product. Flink K8s Toolbox is free software, so why not give it a go :)

NextFractal is an application for creating fractals and other algorithmically generated images. Images are generated from instructions contained in a script. NextFractal provides tools for rendering fractals, creating time-based and event-based animations, organising images, and exporting images and animations.

NextFractal is currently able to interpret two scripting languages: Mandelbrot and CFDG. Mandelbrot is a domain specific language for creating images of the Mandelbrot set and its variants (the Julia and Fatou sets). CFDG is a context-free grammar for creating images by an iterative process.

You don’t need to learn the scripting language to enjoy the examples provided with NextFractal, but if you do, then it is a lot of fun creating your own fractals. But don’t worry, we have created a Wiki to help you, with a tutorial for the scripting language and examples of scripts. The only limit is your imagination.