Skip to content
Muhammed Demirbaş

15 years

Engineering

I am a software engineer in Ankara. Fifteen years in, most of them designing and running distributed systems on the JVM; since late 2023 on big data — the internals of Apache Spark, then Apache Iceberg — and today the Spark + Iceberg storage side of the cloud data platform at Huawei Cloud.

Outside work I ship things: a desktop tool for Iceberg metadata, a statusline for Claude Code, a documentation kit, puzzle games for my kids. I write when something took me a while to understand and would take someone else less.

Open to a few days a month of independent work on data platforms.

Experience

  1. 2023–present

    Huawei Cloud· Senior software engineer, Spark and Iceberg

    The Apache Spark and Apache Iceberg storage side of the cloud data platform; Flink and Paimon where they meet Iceberg.

    • A Kubernetes admission webhook in Go that lets Spark executors use the Vertical Pod Autoscaler, compatible with spark-operator and Volcano and covered by end-to-end tests.
    • A custom Spark cluster manager in Scala and Python as a YARN alternative for Huawei Cloud, and automated token renewal for the internal cache layer.
    • FlinkSQL streaming of row-level changelog events out of Iceberg upsert tables; the upstream merge-scalar-subqueries optimisation brought in; benchmark analysis tooling that makes bottlenecks show up the same way every time.

    Apache Spark, Apache Iceberg, Flink, Kubernetes, Go, Scala, Java, Python

  2. 2017–2023

    Atlassian· Software engineer, distributed systems

    Six years on the systems behind on-call and notifications, through acquisitions, office closures and layoffs; designing distributed systems and keeping them running.

    • A zero-downtime migration of the legacy scheduling system from a monolith to microservices, in use by millions of people worldwide.
    • The distributed on-call event systems, and 99.9% uptime for a global notification service that never stops, on AWS — DynamoDB, SQS, SNS.
    • Fast triage of delivery problems and the technical writing that let the rest of the team fix the next one without me.

    Java, Kotlin, Spring, AWS

  3. 2013–2017

    MilSOFT· Software engineer, defence

    Integration testing for defence systems, where the test tools did not exist yet.

    • Led an in-house Java test and emulation tool from design to deployment; it unblocked integration testing and debugging for other projects.
    • Networking, multithreading, communication protocols, resource limits, logging and custom parsers, each tuned for performance; a 12-bit JPEG parser in C++.

    Java, JavaFX, Swing, C++

  4. 2012–2013

    AnelARGE· Software engineer, network security R&D

    An R&D company doing network security; the glue between stacks that had never met.

    • A WSO2 ESB communication backbone across the company's technology stacks; an Apache web server module in C that routes requests by database criteria; visualisation in Eclipse RCP.

    Java, C, WSO2, Eclipse RCP

  5. 2011

    ExtremePACS· Software engineer, healthcare

    A healthcare startup where we practically lived in the hospitals we built for; many roles at once, procedures still being written.

    • Picture archiving (PACS) and radiology information (RIS) systems in .NET, from requirements to delivery on the hospital floor; internal deployment tools that removed the manual steps.

    C#, .NET, Silverlight, WCF

Selected projects

All projects

Live and in use.

  • Bilge Baykuş

    live

    Daily puzzle games for kids, in Turkish, on the web and as a mobile app.

    Play bilgebaykus.tr

  • havasaha

    live

    A map of where drones may and may not fly in Türkiye — SHGM zones, NOTAM areas, 3D terrain and sun times. No account, no fee.

    havasaha.com.tr

  • Iceberg Lens

    live

    Desktop tool that shows Apache Iceberg and Paimon table metadata as a graph, snapshot by snapshot, instead of reading manifests by hand.

    Website GitHub Read the post

  • claude-quota

    live

    A statusline plugin for Claude Code that shows every usage quota at a glance - the five-hour and weekly windows, model share, spend - so the usage page can stay closed.

    Website GitHub

  • oku

    live

    A documentation kit. Pages are written in Markdown and rendered in the browser by Custom Elements; a CLI builds the multi-page site, a search index, single-file copies for offline reading and an llms.txt for LLM readers.

    Website Documentation GitHub

  • LogMerge

    live

    Merges many large log files and archives into one time-sorted stream; built for throughput on multi-core machines.

    Website GitHub

Recent writing

All posts

Posts on Spark, Iceberg, clean code and the tools I make — the long form of the work above.

  1. Stop Reading Iceberg Manifests by Hand

    A new tool to make Apache Iceberg metadata exploration faster and easier: Iceberg Lens. It turns complex metadata into an interactive graph with detailed inspection, designed for debugging and understanding table structure without manual file parsing.

  2. Design Challenge: Integrate Kubernetes VPA with Apache Spark

    Exploring the challenges and design approaches for integrating Kubernetes Vertical Pod Autoscaler (VPA) with Apache Spark for efficient resource management.

  3. Why I Switched From Backend to Big Data Engineer After 10+ Years

    After over a decade of working as a backend engineer, I switched to a new domain. This post explores my unexpected transition into the world of Big Data and the motivations behind my career shift.