In this article I'll dig into the limitations and benefits of using LLMs for coding and how to bring engineering back to the landscape, which I call vibe engineering. I aim to help readers on a path to vibe engineering, so this is for working engineers, not first-day coders.

In the end you'll realize that it's not about speed, it's about confidence.

Purely empirical, unpretentious, and potentially poetic. Reflections from someone who uses AI daily but doesn't sell courses or posts horror stories. Embrace pragmatism, avoid the radicals, grab a coffee and let's go.

Agenda

• First things first: my journey to agentic coding

• Engineering, Agile & AI

• Vibe (Coding?) Engineering

• My showcase in AI

• Approaching the end, here are my takes

• My 70/30 model: learning and sharpening fundamentals

• Final words

• Disclosure

First things first: my journey to agentic coding

Back in 2022, I was skeptical. ChatGPT 3.5 was peak hype, while most people were testing these new "chat" capabilities. Developers also started looking into it, and I was no different. However, something was off in my mind: asking a probabilistic model to reason about my code was not accurate. Nor right.

The upcoming years went nuts in terms of AI & coding. I tried my best to get rid of AI from my programming routine. While other developers were struggling with ChatGPT to output a lot of spaghetti code, I was crafting Assembly on live streams. AI outputs were never satisfactory. "They" had no access to my codebase to understand and reason with more confidence.

Fast forward to 2025.

Claude Code

In the middle of 2025, I was still coding manually, like an artisan. At every coding task I used to open my NeoVim and get to work.

Then, I looked around and realized I was the only one not using AI. I attended a private talk at my current employer and I asked which AI models I could start using since I'm a NeoVim user and I was not keen on using Cursor or another IDE. The answer was: Claude Code.

It changed everything.

Within a week, I noticed a slight increase in speed, but not the speed boost people talk about. Something was still missing.

It's not about speed, it's about confidence

Soon I realized that using AI, more specifically Claude Code (Sonnet 3.5, at the time), I could deliver with more confidence. Despite still being capable of writing and reading code, an LLM is orders of magnitude faster at retrieving and processing context from any codebase. For instance, my first steps involved reading about best practices of using Claude Code. Also, an LLM is way more capable of writing/dumping huge amounts of code - which of course, brings some tradeoffs.

On the other hand, even with flagship models, an LLM does not reason like humans. I think we have more reasoning capacity, understanding context and requirements more broadly than LLMs.

That said, this combination of leveraging an LLM to get context and provide information so we can reason about it, reflect and decide is indeed a game changer. Hence, I started delivering my tasks with way more confidence with AI assistance.

As a software engineer, I realized I can implement new features, fix bugs, refactor complex codebases and handle more complex tasks that would take me more time. By applying best software engineering and agile practices, I can go beyond, delivering with quality.

But what are those software engineering and agile practices we're talking about?

Engineering, Agile & AI

I'd like to introduce this topic by first talking about Software Engineering. It's not about writing code, it never was, and I'm pretty sure you're sick of hearing this statement many times on the social networks.

Software engineering is about managing complexity through practices that give you confidence in what you build. It involves testing, design, refactoring, code review and all that sort of practices many of us do on a daily basis. We should know when to scope something, when to break it down, when to ship and when to say no.

Yes, sometimes just saying "no" is the right call, but respectful, of course

As for agile, being agile is not about using a tool or some agile framework. Neither about hiring dozens of agile coaches. Truly agile embraces short cycles, working software, responding to change over following a plan. It's a culture.

Everything under an agile culture leans toward feedback loops. Pair programming and retrospectives provide feedback loops. Talking to designers, refining stories, writing spikes, promoting R&D. The whole point is to learn fast, fail fast, and correct course.

Fundamentals matter

Almost everyone in tech has already faced that moment where something breaks in production, and you realize the bug wasn't a lack of tooling, or a missing requirement. It was a gap in fundamentals. Hard to admit that, I know, but the sooner you realize that, the better you get at the software crafting.

When you know the fundamentals, you read code and introspect systems differently. You start having a broader and holistic vision of the whole system. You spot the shortcut before it becomes a regression, and you ask the right question before the wrong code ships.

Got it?

You should ask the right question before the wrong solution lands in production

That's why fundamentals are so important. Even more now, in the AI agentic era. That gut feeling, the one that comes from years of writing, reading, breaking, and fixing code, that's not something an AI model gives you. That's something you earn.

And here's where the irony kicks in: the more capable AI gets at generating code, the more you need fundamentals to work with it, not less. If you can't evaluate what the AI produces, if you can't tell whether the test it wrote actually covers the edge case, if you can't spot the subtle bug in a 3-line diff that looks perfectly clean, then you're not engineering, you're rubber-stamping.

Fundamentals are what turn an LLM's output from a suggestion into a deliberate decision.

Building the bridge

Now here's the thing. LLMs don't bring any of that, because they generate code at scale. They won't tell you if that code is right. Furthermore, they don't validate assumptions, don't catch regressions or question a misleading requirement.

All of that is still on you.

So the question isn't whether AI replaces engineering practices, because it won't. The question is how to use these engineering and agile practices to work with AI, hence increasing quality while delivering solutions to real problems.

A note about SDD and agent orchestration

"Okay Leandro, so you mean we should do SDD and orchestrate dozens of agents?"

Hell no, definitely not. I'm not in this game to overcomplicate things. I think that SDD (or Spec-Driven Development) is fine, but the term is overly inflated and many people are taking SDD like creating a lot of markdown files upfront then orchestrating with dozens of agents.

What documentation enthusiasts are doing with SDD and plenty of agents is actually waterfall. I'm not very fond of waterfall for software development, because in the waterfall model we don't have feedback loops and I'm pretty sure that even with all that documentation upfront and multiple agents that dump code at the speed of light, rework will be needed. Requirements will be missed. Someone will forget important details and the AI won't catch those missing requirements as if it was something magical.

I think our industry evolved for decades to finally get shit done with agile practices and all of a sudden we're doing waterfall again? Please, no.

What I'm going to share here is a simple process with ancient engineering & AI practices that I believe will work for most people in tech, working for startups or even enterprises. We'll dig further into these practices in the upcoming sections.

Vibe (Coding?) Engineering

Vibe coding has been a hot topic lately. Karpathy coined the term, and suddenly a massive number of tech enthusiasts became coders. I'm not against it. I believe that vibe coding genuinely unlocks potential for people who want to build products, people who think in terms of what they want to create, not what syntax they need to type.

I even consider myself a vibe coder despite years of experience writing software for a variety of companies using different programming languages. Not to mention I'm seasoned at different levels of architecture, yet I know I always have a lot of knowledge gaps to fill

But let me bring a new term that I don't see many people talking about: vibe engineering. That's not a rejection of vibe coding, but an enhancement instead.

The practices I'm about to walk through aren't new though, and that's exactly the point. Refine, scope, test, pair, review, debug with discipline. These are the things you should have been doing even before AI, practices that kept software reliable before AI, and they're the stuff that keep AI-generated software reliable now.

The difference is that skipping them used to mean a bug in production. Skipping them with AI in the loop means a bug in production that you didn't even write and worse, you can't explain and reason about.

Let's get into the practices.

Engineering practices for the AI era

This is where most AI coding discussions fall apart. Either you're told to "just prompt harder" or you're told to "never trust the output". Both miss the point. The practices aren't new, and they don't change because agentic AI is here, they just become more important.

Refine before you build

This is non-negotiable. I don't necessarily mean to exaggerate on SDD (Spec driven development) - because I believe people are losing the point about SDD and I can talk about it in another post -, but before touching a keyboard, before opening an agent, you need to know what you're building and why.

I use a skill called "po" (which acts like a Product Owner) that forces me to write a PRD - product requirements document -, before any code. Not because I love documentation, but because the agent will hallucinate scope if I don't carve it myself. A vague prompt gives you vague code, whereas a refined one gives you scoped and testable deliverables. Refine first, then build.

Break work into small deliverables

If your prompt asks for more than a few hundred lines of output, maybe better to step back and break down the requirements. I have a personal threshold: above 500 lines of diff and I start questioning whether the scope is right.

Small diffs are reviewable diffs, which make them easy to test and validate. The agent handles small scoped tasks far better than large ambiguous ones.

For instance, I usually write a prompt to break down in small pieces before I start using the /po skill:

I want to add RSS feed generation to the blog engine.
The feed should produce valid RSS 2.0 XML with:

- channel metadata (title, link, description, whatever else is needed for a RSS)

- items for each published post (title, link, published date, description)

Before writing any code, let's break this into small, independent deliverables.
Give me a roadmap with clear order, where each step produces something 
I can refine, test and validate on its own

The key word here is before. I'm not asking the agent to build anything yet - because coding agents WILL try to build something -, I'm asking it to plan.

Look, I'm not saying anything about the famous "plan mode", I'm not even using it anymore. The output is a mini roadmap, something like:

1. Define the RSS data model (structs for channel and item)
2. Implement XML escaping for special characters
3. Build the feed header (channel metadata)
4. Build the feed items (post to RSS item mapping)
5. Wire it all together and validate against the RSS 2.0 spec

Of course the first output is almost always not ready yet. I refine the ROADMAP. I leave comments before we decide the final version, and iterate over and over until my requirements are satisfied.

Then, once I'm comfortable with the roadmap, I refine one deliverable at a time. I take step 1, run /po to write a proper PRD just for that piece, then /dev, my skill for coding, to implement it.

Test, validate, deliver. What I learn from step 1 might change now I approach step 2, or even reshape the roadmap itself.

Yes, that's the truly agile development. Not about SCRUM, not about KANBAN. That's all about embracing changes. If you are in this game and not keen to embrace changes during the whole cycle of software development, well...I have bad news for you

I don't refine everything upfront, and that's why I'm not very fond of the "SDD" people are selling nowadays. All this SDD thing are a huge pile of requirements that will change, no matter how many agents I have working for me. Completely wasteful on my workflow.

I refine the first thing, ship it, learn, adjust if needed, move to the next one. This is agile applied to AI-assisted work, and it works precisely because the agent performs best on small, scoped tasks. Give it a destination, not a map of the entire country.

Test first, always

TDD is not a methodology debate for me, it's a survival tool. When an LLM writes code, you need a failing test that proves what you expect before the code even exists. Write the test, see it fail, let the agent make it pass, then verify.

This feedback loop, RED -> GREEN -> REFACTOR is nothing new, but it's the closest thing to determinism you'll get from a probabilistic model.

I wrote an article this year about Taming non-determinism: from logic gates to LLMs, in case you want to have a deeper understanding of non-determinism

My skill "dev" relies on TDD from the very start. I invite you to take a look into it.

Pair with the agent

I don't mean to sit back and watch it generate 800 lines. I mean treat the agent like a pair programming partner. You drive, the agent suggests and reviews. Or the agent drives, you suggest and review.

Again, my /dev skill covers different modes for development:

a) agent-pair - two subagents working adversarially, one writes, one gates

b) solo - the agent does the work but narrates every step

c) pair-with-me - me and the agent actually pairing, discussing before coding

At any mode, the skill is ready to work with TDD, no bullshit included. Pick the mode that matches the task, but always stay in the loop.

Review the output

Every. Single. Time. An LLM's output is a suggestion, not a decision. My "review" skill spawns three parallel reviewer agents. One for security, one for performance, and one for quality. Then a red-team auditor that checks the reviewers themselves. It sounds excessive until you've shipped a bug that looked perfectly clean in a 400-line diff. If you're not reviewing, you're rubber-stamping.

Even so, you must review the auditor's final review. Don't take any LLM output as a source of truth. An LLM's output should never be the final destination, but using it with the correct tooling, it'll take you some steps forward.

Debug with discipline

When something breaks, don't just prompt the agent to "fix it". I know, it's tempting to drop a screenshot and ask the LLM to solve it. With a good LLM model and correct skills, this naive prompting will provide the right output very often, but sometimes it simply won't get out of the loop. Humans get stuck, so do AIs.

Tip: reproduce the bug with a failing test first. Actually, that's a practice I've always tried to apply on my workflow way before agentic AI. If there's some skill I'd pick and say that every good engineer should sharpen, it's the ability to reproduce anything with a failing test. Many engineers, including the senior ones, fail at this important step while debugging.

In a distributed system, debugging is very hard. Sometimes a failing test is not enough, but if you spend some time trying to write as many failing tests as possible, you'll eventually get there. The "test-driven" mindset is winner IMHO. "Follow the data and write a failing test for it" is the best engineering advice I've ever heard.

Speaking of AI, I have a "bugfix" skill that enforces this: understand the bug, scout the codebase, write a failing test for the right reason, then fix. No test, no fix. Of course, agents usually write any poor test just to make it pass, but if you enforce it with a good skill, the agent will go to the right path guarded by good engineering practices you provided in the skill or prompt.

The agent will happily patch symptoms all day if you let it. A failing test forces both you and the agent to prove you actually fixed the root cause.

Now time to show what all this looks like in practice.

My showcase in AI

I want to highlight three real use cases where I applied those engineering practices and could be successful with it. Despite using AI for personal projects, I've been using agentic AI for delivering value to real customers in production, critical projects and demanding features. It's not a playground as many may argue: it's serious work.

Case 1: Debugging a PDF rendering bug

I was working on a task where customers weren't able to download some PDFs generated by the platform, affecting critical workflows in production. Pay attention to the "some PDFs" part. Not all of them.

Logs pointed that some PDFs were failing to parse, leading the troubleshooting to go look at broken PDFs. In the "PDF world", everything is messy. Writer and reader tools don't always come to an agreement, despite the PDF itself having a public specification.

I got some broken PDFs and started looking into them, as they were opening normally in my PDF reader. The problem was somewhere in our merge PDF process, which is backed by Ghostscript, a well-known tool for printing and PDF manipulation.

With this information in place, instead of pasting the bug report into an agent and saying "fix it", I called my /bugfix skill. And you'll see how a good skill and a failing test win.

The skill started by understanding the bug: what's broken, when it happens, what's the expected behaviour. Then it scouted the codebase, reading the PDF parsing code, tracing data flow, checking git history for recent changes.

It came back with a hypothesis: the outline builder within the PDF content was pushing a parent reference into its own children, creating a cycle.

The critical step: write a test that fails. The skill produced a minimal test case that created a PDF outline with two levels, asserted that traversing the outline tree terminates, and the test failed, proving the bug existed.

Once we can prove the bug exists with the right failing test, the dopamine levels skyrocket :P

The "senior" moment

First, I let the AI go for a fix, but as soon as I noticed it got stuck in a loop after two or three failed approaches, I decided to enter the loop and become a real senior. My instinct was that something in the Ghostscript process was not recognizing the circular outlines. But Ghostscript is an external tool that I don't have control over. The only thing I could do was to try a variety of different command line flags in a "trial-and-error process". Not good.

The senior moment became when I asked the agent to analyze the Ghostscript process using some tooling in my macOS, which is an operating system based on BSD.

For freak's sake, the agent pulled out the utility sample against the gs process and then got the process's call stack.

And boom! My little boy got the exact part of the Ghostscript stacktrace where the circular reference was stuck in an infinite loop. The fix was just using the correct flag to "ignore" circular outlines. Easy peasy.

Could I hit the same destination without AI? Hell yes, but I would have spent too many hours on it, maybe days. The whole process took me only 20 minutes. More importantly, the bug is now captured in a test. If it ever regresses, I'll know immediately.

Case 2: Building a feature from scratch, product to code

ChessWAV is a Rust project that turns chess games into audio. It's not a simple CRUD. It's chess logic, graph manipulation, sine waves, mathematical stuff and all. The REPL had a working board display, but it was appending a new board after every move, scrolling the terminal endlessly. No move history, no file labels on both sides. I wanted to fix that.

Instead of jumping to code, I started with /po in-place board rendering with move list sidebar. The skill walked me through writing a proper PRD: what the rendering should look like, how ANSI cursor escapes work for in-place updates, how each display strategy (sprite, unicode, ascii) affects layout calculations. Edge cases like switching display modes mid-game and cursor-up calculations across different board heights.

In case you don't believe me and think I could be making this up, please refer to my issue in Github, the pull request, all of it done during a live stream (pt-BR) in my Youtube channel

The PRD took maybe ten minutes and saved me from scope creep on what could have been a bottomless rendering rabbit hole. Of course I had two or three iterations before I agreed on the PRD, but that's the true productivity power of using agentic AI IMHO.

After the issue was created, I handed it to /dev, no further prompting, just pasted the issue URL, then it scouted the codebase, asked clarifying questions about the existing display strategy pattern, proposed test cases, and only after did it start coding.

Every cycle was RED then GREEN. The whole feature shipped in three incremental diffs: cursor-up logic and layout height calculation first, then the move list sidebar, then the re-rendering flow that ties it together.

Before opening the PR, I ran the /review skill which gloriously gave me some hints on refactoring. I looked at the code afterwards, though it wasn't too bad, I suggested some refactoring to make the code cleaner. You can see the code yourself, as the final PR was only 369 additions across 3 files. Not 2000 lines of "here's the whole thing". Small, reviewable, tested increments. Each one I could understand, validate and ship with confidence.

And before someone argues:

But Leandro, you're doing engineering as usual. Why the need of using AI at all?

Let's be honest here. I'm a seasoned software engineer and I'm pretty sure I couldn't deliver this kind of complexity in a matter of minutes or hours with confidence. That's not AI-slop, go check yourself. This use case is the difference between prompting "add in-place board rendering" and engineering your way to it.

I kindly invite you to truly experiment on this flow. Give engineering & AI an honest chance.

Unless you're a radical who's not keen to learn and evolve. If that's the case, well...you shouldn't be reading my blog at this point. Feel free to close this tab now. Radicals are not welcome here.

Case 3: Review before you ship

I had just finished a refactor of the markdown rendering pipeline for DevTUI, the blog engine I created, using it to write this very blog post. The diff was around 300 lines, which is already pushing my comfort zone. I ran /review on the changes.

The skill pulled the diff, launched a scout to understand the surrounding code, then spawned three parallel reviewers. Security checked for injection risks in the HTML output. Performance looked for unnecessary allocations in the hot path. Quality examined whether the refactor actually improved the design or just moved code around.

Then the red-team auditor cross-checked all three reports, catching a false positive from the security reviewer and flagging a blind spot the others missed: the refactor had changed the order of markdown preprocessing steps, which would silently break nested blockquotes on certain inputs.

I would not have caught that by skimming the diff. The fix was two lines, but finding it required reading the diff against the full context of how the pipeline processes input.

That's what review gives you, not just "looks good" - which many developers already do -, but actual evidence that your change does what you think it does.

Approaching the end, here are my takes

Everything I described above, the skills, the TDD discipline, the review workflow, it's not magic. It's configuration and intentionality. Next are a few things from my setup that make this work. Also comes with a few takes, read it with discretion.

Write your own CLAUDE.md (or AGENTS.md)

This is the most impactful single thing you can do. My personal ~/.claude/CLAUDE.md (check the correct configuration and conventions for your harness) is my development philosophy distilled into a file the agent reads on every session. It should be short. It says things like "deletion beats addition", "TDD for non-trivial work", "search the codebase first", "baby steps". Without it, the agent follows its own defaults, and those defaults are optimized for generating output, not for engineering.

Write yours, be opinionated. The more specific, the better.

Create skills for your recurring workflows

My /po, /dev, /bugfix and /review skills encode processes I actually follow. Not theoretical best practices, but the way I work. A skill is just a markdown file that tells the agent what to do, step by step, with gates and waits. If you find yourself repeating the same process, write it down once and let the skill enforce it. That's how you get TDD from a model that defaults to dumping code.

And that's the main reason I avoid importing other's skills I don't fully understand or control. I avoid using repos with thousands of skills or "following some framework". I think we should pick up skills with caution. Always better to start from a reference and then craft a few of your own.

Feel free to pick up my skills, but use your agent to explain about them, refine and adapt to your own workflow.

Set up hooks for flow state

I have a shell hook that plays a sound and flashes my Tmux window when the agent finishes, so I don't sit staring at a terminal waiting. Another hook loads context from my Obsidian vault at session start, so the agent already knows my project history. Small things that compound.

Use adversarial agents

My /dev skill runs two subagents in pair mode: a driver that writes code and a navigator that gates every step. The navigator doesn't rubber-stamp. It pushes back on test scope, challenges minimum implementations, flags premature abstractions. This costs a bit more tokens, I know, but saves you from the model's tendency to over build.

One agent writing unchecked code is vibe coding. Two agents checking each other is closer to pair programming.

It's not just Claude

I've been running GLM 5.1 and Kimi 2.6 for personal projects and I'm genuinely impressed. Both are strong models that hold their own for coding tasks. The important thing is that my skills, my CLAUDE.md, my memory, they're all model-agnostic configuration files. I can switch models and the process stays the same.

I've been using opencode and pi-agent alongside Claude Code, and both harnesses read from the same ~/.claude configuration. Same skills, same rules, same workflows. Pi-agent works with extensions instead of Claude's hook system, so there are slight differences, but the core setup translates. They are way cheaper too.

The discipline is in the configuration, not in the model. Pick whatever works for your budget and preference, but bring your own process.

AI only reflects your capability

I'm going to be blunt here. The quality of AI-assisted work is directly proportional to the quality of the person guiding it. If you don't know how to write a test, the agent won't write a meaningful test. If you don't know how to review a diff, the agent's output passes your eyes without triggering a single alarm. If you don't understand your system's architecture - and here's the catch -, you can't tell the agent where to look, and it will simply guess. Sometimes right, often wrong, and you won't know which is which.

This is not a flaw in AI. It's a mirror. The agent amplifies what you bring. Bring fundamentals, and you get leverage. Bring ignorance, and you get confident ignorance at scale.

That's why I keep my craft projects. Assembly, Rust from scratch, neural networks without frameworks. Not because I'm nostalgic, but because that's where I build synaptic connections that make me deliver better with AI.

Skip it, and you're not vibe engineering. You're vibe hoping.

My 70/30 model: learning and sharpening fundamentals

Seventy percent AI-assisted work, shipping, delivering. Thirty percent manual coding, learning, building intuition. The 30% is what makes the 70% work. Without fundamentals, you become a prompt engineer who can't debug. With only fundamentals, you ship slower than the field. Why not both?

Again, avoid the radicals.

"But how to keep learning and sharpening the fundamentals?"

How do I keep the 30% effective? How do I code, learn, and get intuition from the decisions? Ironically, the answer is in the LLM itself. Yes, we can leverage the LLM as a pair programming partner, remember the topic from earlier?

When I'm coding manually, I leave the agent open reading my code, doing TDD with me, where I'm the driver, the agent is the navigator. It watches my changes and provides feedback in real-time, which I can reason about, get intuition and iterate until I get a solution that I fully understand. Whenever I feel I'm good with the current development, the rest is only ceremony and boilerplate, then I switch to vibe coding and the agent finishes the boring work.

My skill /dev also covers this mode, which I call pair-with-me. Every time I want to code and learn something new, I'm using this mode. Otherwise, I'm doing engineering around agents as I already distilled throughout this whole article.

Here's another live stream where I'm studying using this technique (pt-BR), which I use daily too.

Final words

I started this article saying I'd been procrastinating on writing it. Now I understand why. This isn't a hot take or a framework pitch. It's the distillation of months of trial and error, of shipping real features to real customers, of getting stuck in agent loops and finding my way out.

The message is simple: vibe coding is real and it's powerful. But vibe coding without engineering is just faster ways to produce code you can't explain, debug or maintain.

Vibe engineering is what happens when you take that speed and pair it with discipline. Not the kind of discipline that slows you down, but the kind that makes speed sustainable.

Yet I can't understand why so many "seniors" struggle to see this simple conclusion

And for the love of whatever you believe in, don't skip the fundamentals. Any content about software architecture is always the best option. Look, I'm not saying anything about SOLID or Design Patterns, because IMHO those are just coding design, not software architecture. Safe to skim them.

I suggest you find your path to master how systems are created and how they communicate to each other. That's the sweet spot for software engineering, and that's an ongoing ability we're never done with.

Books I recommend

If there's one thing I learned about reading tech books, it's that most of them age like milk. Framework-specific books, language tutorials, "building X with Y", they're outdated before the ink dries. I always avoid those.

What I look for are books that are timeless. Books about principles, tradeoffs, and how systems think, not configuration tutorials or one random opinion from a guy from Arizona. You read timeless books once and they keep paying dividends years later because the problems they address haven't changed. We still distribute systems, communicate between services and deal with complexity and uncertainty.

Here are five that I keep coming back to:

• The Mythical Man-Month, Fred Brooks. This book is from 1975 (yup, that old) and it's still right. Adding people to a late project makes it later. Communication overhead grows quadratically. There's no silver bullet.

• The Pragmatic Programmer, Hunt & Thomas. This book taught me that programming is not about syntax, it's about thinking. DRY, tracing bullets, debugging by thinking. It's practical without being tied to a language or framework. Timeless.

• Fundamentals of Software Architecture, Richards & Ford. This goes straight to the sweet spot I mentioned. Architecture patterns, tradeoffs, modularity, how to make decisions when there's no right answer. The chapter on tradeoff analysis alone is worth the whole book. You read it and you'll never be the same developer.

• Architecture the Hard Parts, Richards, Ford, Mishra & Sadalage. The sequel in spirit. Where most architecture books show you the happy path, this one shows you the distributed transactions. This is the book that teaches you how systems communicate with each other and why that's where the real complexity lives.

• Extreme Programming Explained: Embrace Change, Kent Beck. The book that gave agile its heart. Not the agile of frameworks and certifications, the agile of short cycles, continuous feedback, and actually embracing change instead of pretending you can plan it away. Everything I said about feedback loops, small deliverables, test-first, pair programming, Kent Beck was writing about it in 1999. Embrace change.

These five books, together, cover exactly what I mean by the sweet spot. How systems are created, how they communicate, how to manage complexity, and how to make decisions under uncertainty. Read them in any order, but read them.

And for the best...

Software engineering is dead. Long live software engineering.

Disclosure

This article was written using a TUI editor I built in Rust called DevTUI, with the support of GLM 5.1 via pi-agent for drafting and context.

Every single paragraph came from my head. The LLM never wrote a sentence for me. It questioned, suggested, challenged, and I decided. Much like I use it daily.

The identity, opinions, takes and bad jokes in this article are 100% mine. The AI was a pair, not a ghostwriter.

O assunto subiu para o trending topics global e os gringos ficavam se perguntando:

Why is palindrome such a hot topic in Brazil now?

Mal sabem eles que aqui no Brasil TUDO vira meme, até mesmo o "67" deles. Melhoramos o "67" e escalamos para um nível diferenciado. Google pesquisar.

Não vou entrar em detalhes, mas basicamente a "treta" foi criada porque alguém mencionou que não conseguia contratar desenvolvedores pois nenhum candidato passava no teste do palíndromo. O negócio escalou. Teve implementação de palíndromo de todos os tipos e também teve muito meme, que foi a melhor parte pra mim.

Mas vamos lá, quem acompanha meu blog sabe que não escrevo por escrever. Eu gosto de explorar um assunto técnico e ir fundo (lá ele), captando e compartilhando conhecimento no meio do caminho. É pra isso que estou aqui.

Agenda

• First things first

• Palindromaniacs

  • Mas a gente é diferente: UNIX signals + queue + stack

• UNIX signals como forma de comunicação

  • Enviando binário com dois UNIX signals
  • Montando bytes a partir dos bits
  • Enviando strings inteiras

• Marcando o fim da mensagem

• Fila e pilha: o teorema central

  • Conectando tudo

• Bonus points: Handshake FTW

• Conclusão gloriosa

First things first

E a pedido de muitos, incluindo o brabíssimo Navarro, vamos fazer uma implementação do palíndromo em Bash:

Por que Bash, Leandro?

Porque entre todos os que submeteram soluções para a 1ª Rinha de Backend e as subsequentes, eu fui o primeiro da linhagem com uma submissão funcional porém nada performática.

O que era pra ser um meme no Twitter, acabou virando o apelido "carinha do Bash". É sobre isso.

Palindromaniacs

Uma das implementações mais comuns para palindromania chama-se two pointers, onde definimos dois ponteiros na string e vamos comparando o primeiro com o último, o segundo com o penúltimo e assim por diante, mexendo nos ponteiros até chegar na metade da string. Se todas as comparações derem certo, a palavra é palíndromo.

Em Bash dá pra implementar isso de forma mais legível usando substrings, que é prima do two pointers:

is_palindrome() {
    local input="$1"
    local length=${#input}

    for ((i=0; i<length/2; i++)) do
        local leftmost=${input:$i:1}
        local rightmost=${input:$((length-1-i)):1}

        [ "$leftmost" != "$rightmost" ] && return 1 # Error status code, IS NOT a palindrome
    done

    return 0 # Success status code, IS a palindrome
}

Sem entrar muito em detalhes, vou explicar pois Bash:

• local input="$1": joga o primeiro argumento da linha de comando na variável input

• local length=${#input}: pega o tamanho do input

• for ((i=0; i<length/2; i++)): itera sobre cada caractere da string até a metade

• local leftmost=${input:$i:1}: pega 1 caractere a partir da posição $i, ou seja, o mais a esquerda possível

• local rightmost=${input:$((length-1-i)):1}: pega 1 caractere a partir da posição length-1-i, que vai andando do último pra dentro conforme i cresce, ou seja, o mais a direita possível

• [ "$leftmost" != "$rightmost" ] && return 1: compara os dois, se forem diferentes, a palavra não é um palíndromo

Caso não encontre nenhuma divergência nas comparações, então a palavra é um palíndromo, portanto return 0 com sucesso.

Outras implementações que podemos encontrar:

• inverte a string e compara: essa é mais naive, geralmente one-liner e não requer muitos neurônios

• recursão: mais idiomática em linguagens funcionais

• stack: empilha a primeira metade, depois lê a segunda metade desempilhando e comparando a cada caractere

• e mais: deque (double-ended queue), programação dinâmica, etc tal e coisa

Mas a gente é diferente: UNIX signals + queue + stack

Implementar palíndromo no modo tradicional faria Bash e este post serem mais do mesmo. Não tem graça, Bash precisa ser requintado. Então resolvi ser o diferentão inovar e resolver este problema utilizando apenas dois sinais UNIX, uma fila e uma pilha. A ideia, em três passos:

• um sender que manda a string bit a bit via signals

• um receiver que monta cada caractere e empilha em duas estruturas (FIFO e LIFO)

• no final, drena as duas estruturas. Se todo pop da pilha bater com todo dequeue da fila, é palíndromo.

FIFO significa first-in, first-out. É a nossa fila. E LIFO significa last-in, first-out, neste caso nossa pilha. Tudo nosso.

Basicamente, estamos explorando duas estruturas de dados primordiais da computação para verificar se uma palavra é palíndromo. E não só, estamos enviando a string utilizando IPC de uma forma bastante primitiva, com o envio de sinais.

Se quiser saber mais detalhes sobre UNIX signals e como os utilizei para construir um pequeno broker de mensageria, veja meu artigo You don't need Kafka: building a message queue with only two UNIX signals. Lá eu explico o necessário sobre envio de sinais e também como implementei a modulação/demodulação de sinais para bits.

UNIX signals como forma de comunicação

O intuito deste artigo não é aprofundar muito nos sinais UNIX, uma vez que já abordei com profundidade no polêmico artigo anterior. Mas a seguir temos um exemplo bastante simples para estarmos na mesma página:

#!/usr/bin/env bash

trap 'echo "Received signal SIGUSR1"' SIGUSR1

echo "PID: $$"
echo "Rodar em outra sessão shell: kill -SIGUSR1 $$"

while true; do
    sleep 0.5 || true
done

$ bash receiver.sh
PID: 54112
Rodar em outra sessão shell: kill -SIGUSR1 54112

Em outra sessão:

$ kill -SIGUSR1 54112

Com isto, receiver imprime Received signal SIGUSR1 pra cada vez que enviarmos o sinal!

O que é um sinal, afinal? Pra quem nunca ouviu falar, signals são a forma mais primitiva de IPC no UNIX. Você pode mandar kill -SIGTERM num processo pra encerrá-lo, kill -SIGINT pro mesmo efeito de Ctrl+C, e por aí vai. A graça é que o processo que recebe pode interceptar esses signals e fazer o que quiser. Esta é a famosa técnica de signal trap.

Nossa, Leandro, mas o kill não "mata" o processo?

Não, ele serve para envio de sinais, incluindo os clássicos SIGTERM e SIGKILL. Reforço: veja meu artigo de UNIX signals para entender as nuances.

Enviando binário com dois UNIX signals

Quando falamos de IPC primitivo tal como UNIX signals, não conseguimos enviar uma "mensagem" no formato que estamos acostumados (string). Nem bytes. Receber um signal é como receber um boolean: ligou ou não ligou. É a representação de apenas dois estados possíveis.

O que UNIX nos entrega são dois sinais user-defined, SIGUSR1 e SIGUSR2, livres para usarmos como quisermos. Não conseguimos enviar bytes, mas temos a primitiva necessária para representar um byte: bits.

• SIGUSR1 = bit 0

• SIGUSR2 = bit 1

Pronto, acabamos de inventar uma linguagem binária. O receiver agora fica assim:

  #!/usr/bin/env bash

  trap 'echo -n "0"' SIGUSR1
  trap 'echo -n "1"' SIGUSR2

  echo "PID: $$"
  echo "Rodar: kill -SIGUSR1 $$ (bit 0) ou kill -SIGUSR2 $$ (bit 1)"

  while true; do
      sleep 0.5 || true
  done

$ bash ./receiver.sh
PID: 73251
Rodar: kill -SIGUSR1 73251 (bit 0) ou kill -SIGUSR2 73251 (bit 1)

# Em outra sessão:
$ kill -SIGUSR1 73251
$ kill -SIGUSR1 73251
$ kill -SIGUSR1 73251
$ kill -SIGUSR2 73251
$ kill -SIGUSR2 73251
$ kill -SIGUSR2 73251
$ kill -SIGUSR1 73251
$ kill -SIGUSR1 73251

O que imprime 0001 1100, seja lá o que isso significa :)

Agora a gente fala binário. Cada kill é um bit chegando no fio. Mas só bit solto não vira muita coisa. 8 bits formam um byte, e cada byte representa um caractere ASCII. Por exemplo, a letra A tem código ASCII 65, que em binário é 0100 0001.

Se conseguirmos receber esses 8 bits e transformar isso em caractere, conseguimos transmitir uma string inteira, um caractere por vez.

Montando bytes a partir dos bits

Já sabemos que 8 bits formam 1 byte. E que cada byte representa 1 caractere ASCII. Pra transformar a chuva de SIGUSR1 e SIGUSR2 em texto, o receiver precisa acumular os bits e, a cada 8, converter em caractere.

Vamos usar uma convenção onde o primeiro bit que chega é o de menor peso (bit 0), o segundo é o bit 1, o terceiro é o bit 2, e assim por diante até o bit 7. Ou seja, vamos utilizar LSB, ou least significant bit, que traduzido é "bit menos significativo".

  #!/usr/bin/env bash

  POSITION=0
  ACCUMULATOR=0

  decode_bit() {
      local bit=$1
      ACCUMULATOR=$(( ACCUMULATOR + (bit << POSITION) ))
      POSITION=$(( POSITION + 1 ))

      if [ "$POSITION" -eq 8 ]; then
          local char
          char=$(printf "\\$(printf '%03o' "$ACCUMULATOR")")
          echo "byte: $ACCUMULATOR | char: $char"
          POSITION=0
          ACCUMULATOR=0
      fi
  }

  trap 'decode_bit 0' SIGUSR1
  trap 'decode_bit 1' SIGUSR2

  echo "PID: $$"

  while true; do
      sleep 0.5 || true
  done

Explicando as partes que mais importam:

• (bit << POSITION): shift à esquerda. O bit é empurrado pra posição certa dentro do byte. Se POSITION é 6 e o bit é 1, isso vira 64 (porque 1 << 6 = 64). Se o bit é 0, fica sempre 0.

• quando POSITION chega em 8, fechamos o byte. Convertemos pro caractere com printf, imprimimos e zeramos tudo pro próximo byte

Vamos ver isto funcionando na prática: A letra A tem código ASCII 65 como já vimos, que em binário é 0100 0001. Mandando LSB primeiro, a sequência de bits vira 1 0 0 0 0 0 1 0:

$ bash receiver.sh
PID: 80381

# Em outra sessão:
$ kill -SIGUSR2 80381   # 1
$ kill -SIGUSR1 80381   # 0
$ kill -SIGUSR1 80381   # 0
$ kill -SIGUSR1 80381   # 0
$ kill -SIGUSR1 80381   # 0
$ kill -SIGUSR1 80381   # 0
$ kill -SIGUSR2 80381   # 1
$ kill -SIGUSR1 80381   # 0

O receiver imprime: byte: 65 | char: A. What a wonderful day, uh?

Signals viraram bits. Bits viraram byte. Byte virou caractere. Agora dá pra mandar palavras inteiras, onde cada caractere é apenas uma sequência de 8 bits, e o receiver decodifica um por um conforme chegam.

Manda mais bytes para ver outras letras: B = 66 = 0100 0010 e assim por diante. Não esquecendo de inverter a ordem dos bits, do LSB pro MSB.

Nota: poderíamos utilizar o formato MSB (most-significant bit), para não precisarmos inverter os bits no envio, mas pra deixar o código do receiver mais simples e seguindo a maioria das convenções, decidi utilizar LSB mesmo

Enviando strings inteiras

No momento só conseguimos mandar bits "na mão", utilizando kill -SIGUSR1 ou kill -SIGUSR2, um por um. Porém isso não escala. Precisamos do outro lado da história, um sender que recebe uma string e cuida de mandar os bits certos.

A lógica é o "espelho" do receiver basicamente:

• pra cada caractere da string, pega o código ASCII

• codifica para bits (8 bits), seguindo convenção LSB

• manda cada bit como SIGUSR1 (0) ou SIGUSR2 (1)

sender.sh

  #!/usr/bin/env bash

  # usage: ./sender.sh <pid> <string>
  PID=$1
  INPUT=$2

  send_bit() {
      if [ "$1" -eq 0 ]; then
          kill -SIGUSR1 "$PID"
      else
          kill -SIGUSR2 "$PID"
      fi
      sleep 0.5
  }

  send_byte() {
      local byte=$1
      local k
      for ((k=0; k<8; k++)); do
          send_bit $(( (byte >> k) & 1 ))
      done
  }

  for ((i=0; i<${#INPUT}; i++)); do
      char="${INPUT:$i:1}"
      code=$(printf '%d' "'$char")
      send_byte "$code"
  done

O core está em send_byte:

• (byte >> k) & 1: pega o k-ésimo bit do byte. Faz shift à direita por k posições e aplica & 1 (máscara) para isolar o bit. Em k=0, pega o LSB, em k=7, pega o MSB.

• sleep 0.5 entre bits: signals podem ser perdidos se chegarem rápido demais (o sistema operacional colapsa sinais idênticos pendentes, como forma de "debounce"). Esse delay dá tempo do receiver processar cada um antes do próximo chegar

Conseguimos mandar uma string inteira via signals. Cada caractere viajou como 8 bits via IPC, foi remontado byte a byte, e o receiver imprimiu conforme decodificava. Vamos provar isso na próxima seção, onde também resolvemos o fim de mensagem.

Nossa, isso é mágica?

Não, é apenas trap + bit shift + acumulador.

Entretanto, temos um problema. O receiver não sabe quando a mensagem acabou. Ele fica esperando o próximo bit pra sempre, sem poder fazer nada com a string completa.

Marcando o fim da mensagem

O receiver fica eternamente esperando bits. A gente precisa avisar quando a string acabou. Uma convenção clássica em comunicação binária é o famoso byte zero (0x00, conhecido como NUL). Isto marca o fim da mensagem. Strings em C usam essa ideia há décadas, então vamos roubar.

No sender, basta mandar 8 bits "zeros" logo depois do último caractere:

  # ...todo o loop de envio dos caracteres...

  # fim da mensagem
  send_byte 0

E no receiver, quando o byte completo for 0, em vez de decodificar como caractere, marcamos o fim:

  if [ "$POSITION" -eq 8 ]; then
      if [ "$ACCUMULATOR" -eq 0 ]; then
          echo ""
          echo "[end of message]"
      else
          local char
          char=$(printf "\\$(printf '%03o' "$ACCUMULATOR")")
          echo -n "$char"
      fi
      POSITION=0
      ACCUMULATOR=0
  fi

Vamos rodar tudo:

# Receiver
$ bash receiver.sh
PID: 15300

# Sender
$ bash sender.sh 15300 ana

# Receiver
ana
[end of message]

Cool, uh? Agora temos a base completa de transporte. Bits viram bytes, que viram caracteres, e o NUL fecha a mensagem. Tudo isso com apenas dois sinais e um trap.

Contudo, falta a peça principal do post: a lógica do palíndromo, implementada via fila e pilha conforme prometemos.

Fila e pilha: o teorema central

Vamos esquecer signals por um momento. Esquece também substrings, índices, two pointers. Imagina que os caracteres da string chegam um a um, e que precisamos decidir se formam um palíndromo, sem precisar remontar a string em memória. É aqui que está o sumo deste artigo! Os signals foram apenas uma "armadilha" que montei pra você, caro leitor.

Tô brincando, eu quis fazer um trocadilho com "trap". Espero que a piada tenha servido de alguma coisa

Como todo engenheiro de software, temos que fazer a pergunta certa: que estruturas de dados, sozinhas, decidem palíndromo por construção? Resposta: uma fila (FIFO) e uma pilha (LIFO).

Uai, por quê?

Pensa no que cada uma faz:

• FIFO: preserva a ordem. Enfileirei R, A, D, A, R, desenfileiro R, A, D, A, R. Mesma ordem.

• LIFO: inverte a ordem. Empilhei R, A, D, A, R, desempilho R, A, D, A, R. Ordem reversa.

E palíndromo é, por definição, uma string que lida na ordem normal e na ordem reversa dá a mesma sequência. Que é exatamente o que acontece se a saída da fila bater com a saída da pilha.

Supa hot fireeee

Em outras palavras, uma string é palíndromo se e somente se, ao empilhar e enfileirar cada caractere em paralelo, todo pop da pilha for igual ao dequeue da fila. Sem comparação posicional, nem length / 2, nem indexação adicional. As duas estruturas decidem por construção. Puro suco da computação ao nosso favor.

palindromer.sh

  #!/usr/bin/env bash

  # usage: ./palindromer.sh <string>
  INPUT=$1

  stack=()
  queue=()

  # push and enqueue each char
  for ((i=0; i<${#INPUT}; i++)); do
      char="${INPUT:$i:1}"
      stack+=("$char")
      queue+=("$char")
  done

  # flush both and compare
  while [ "${#stack[@]}" -gt 0 ]; do
      if [ "${stack[-1]}" != "${queue[0]}" ]; then
          echo "false"
          exit 0
      fi
      unset 'stack[-1]'
      queue=("${queue[@]:1}")
  done

  echo "true"

• stack+=("$char"): append em array é o push da pilha

• queue+=("$char"): o mesmo append serve como enqueue da fila

• ${stack[-1]}: pega o topo da pilha (último elemento, last-in)

• ${queue[0]}: pega a frente da fila (primeiro elemento, first-in)

• unset 'stack[-1]': remove o topo (pop)

• queue=("${queue[@]:1}"): reatribui a fila a partir do segundo elemento (dequeue)

Rodando tudo:

$ bash palindromer.sh ANA
true

$ bash palindromer.sh ABC
false

$ bash palindromer.sh RADAR
true

Temos então palíndromo com uma fila, uma pilha, e a propriedade matemática que cada uma representa. Agora nos resta plugar tudo isso no receiver. Cada caractere que chegar via signals vai direto pras duas estruturas, e quando o NUL chegar, drenamos e respondemos.

Conectando tudo

A função do palindromer.sh recebia uma string como argumento. Aqui, a string não existe inteira em lugar nenhum. Os caracteres chegam, um a cada 8 bits, conforme o sender vai mandando. Mas a lógica é exatamente a mesma.

Em vez de iterar sobre uma string já pronta, a gente empilha e enfileira conforme cada caractere chega. Incrível, não? Desta forma o receiver muda um pouco de lógica:

• cada vez que um byte completo é decodificado e não é o NUL, a gente empilha e enfileira o caractere

• quando o NUL chega, fazemos a drenagem das duas estruturas e respondemos com true ou false

  #!/usr/bin/env bash

  POSITION=0
  ACCUMULATOR=0
  stack=()
  queue=()

  drain_and_decide() {
      while [ "${#stack[@]}" -gt 0 ]; do
          if [ "${stack[-1]}" != "${queue[0]}" ]; then
              echo "false"
              stack=()
              queue=()
              return
          fi
          unset 'stack[-1]'
          queue=("${queue[@]:1}")
      done
      echo "true"
  }

  decode_bit() {
      local bit=$1
      ACCUMULATOR=$(( ACCUMULATOR + (bit << POSITION) ))
      POSITION=$(( POSITION + 1 ))

      if [ "$POSITION" -eq 8 ]; then
          if [ "$ACCUMULATOR" -eq 0 ]; then
              drain_and_decide
          else
              local char
              char=$(printf "\\$(printf '%03o' "$ACCUMULATOR")")
              stack+=("$char")
              queue+=("$char")
          fi
          POSITION=0
          ACCUMULATOR=0
      fi
  }

  trap 'decode_bit 0' SIGUSR1
  trap 'decode_bit 1' SIGUSR2

  echo "PID: $$"

  while true; do
      sleep 0.5 || true
  done

Diferença pro receiver da versão anterior:

• duas variáveis globais novas: stack=() e queue=()

• quando o byte completa, em vez de só imprimir o caractere, a gente manda pra stack e queue

• quando o byte é NUL, em vez de só sinalizar o fim, disparamos o drain_and_decide que devolve o veredito

Momento da verdade:

# Receiver
$ bash receiver.sh
PID: 33485

# Sender
$ bash sender.sh 33485 ANA
$ bash sender.sh 33485 ABC
$ bash sender.sh 33485 RADAR
$ bash sender.sh 33485 SOCORRAMMESUBINOONIBUSEMMARROCOS

# Receiver
true
false
true
true

OMFG! Cada string é codificada via signals, e depois a lógica do palíndromo é feita sem remontar a string novamente!

Bonus points: Handshake FTW

Até aqui o receiver imprime true/false no terminal dele. O sender não fica sabendo de nada. Vamos fechar este ciclo com o receiver devolvendo um signal para o sender imprimir localmente.

Convenção: SIGUSR2 = palíndromo (true), SIGUSR1 = não é palíndromo (false).

O receiver precisa saber quem é o sender. Sender escreve o próprio PID em /tmp/sender.pid antes de mandar a string, e configura um trap para receber o veredito:

# sender.sh
echo $$ > /tmp/sender.pid

RESPONSE=""
trap 'RESPONSE=0' SIGUSR1
trap 'RESPONSE=1' SIGUSR2

# ... send the string ...

# after EOF, RESPONSE was set from verdict
[ "$RESPONSE" = "1" ] && echo "true" || echo "false"

Do lado do receiver, o drain_and_decide agora termina enviando o signal de volta:

  verdict() {
      local result=$1
      local pid
      pid=$(cat /tmp/sender.pid 2>/dev/null)

      if [ "$result" = "true" ]; then
          kill -SIGUSR2 "$pid" 2>/dev/null
      else
          kill -SIGUSR1 "$pid" 2>/dev/null
      fi
  }

Com isso, sender chama, recebe o veredito via signal, e imprime. Ciclo fechado.

Mas isso tá lento demais, o sleep 0.5 por bit dá uns 16 segundos só pra ANA. No caso do "SOCORRAMMESUBINOONIBUSEMMARROCOS", demorou bons minutos...inaceitável

Uma forma de contornar o delay é através de um mecanismo de handshake. O receiver responde com um ACK depois de cada bit, e o sender só manda o próximo quando recebeu. Como o sender só avança após o ACK, nunca há dois sinais enfileirados, portanto a coalescência é eliminada por construção. Sem fila de sinais pendentes, e sem sleep arbitrário.

# receiver.sh

  ack() {
      local pid
      pid=$(cat /tmp/sender.pid 2>/dev/null)
      [ -n "$pid" ] && kill -SIGUSR1 "$pid" 2>/dev/null
  }

E o sender:

  # sender.sh — espera o ACK no lugar do sleep
  send_bit() {
      RESPONSE=""
      if [ "$1" -eq 0 ]; then
          kill -SIGUSR1 "$RECEIVER_PID"
      else
          kill -SIGUSR2 "$RECEIVER_PID"
      fi
      while [ -z "$RESPONSE" ]; do
          sleep 0.001 || true
      done
  }

O detalhe que muda tudo: o loop ocioso do receiver também precisa ser tight. Trocamos sleep 0.5 por sleep 0.001 no while true principal. Sem isso, cada round-trip pagava parte do meio segundo.

  # receiver.sh
  while true; do
      sleep 0.001 || true
  done

Rodando agora:

$ time bash sender.sh 19204 ANA
true
real    0m0.164s

De 16 segundos para 160 milissegundos. 100x mais rápido!!!!11. How amazing is that?

Conclusão gloriosa

A treta do palíndromo virou desculpa pra brincar com duas coisas que eu gosto: UNIX signals e estruturas de dados clássicas. O resultado é isso que acabamos de ver, dois processos conversando por sinais primitivos, e duas estruturas primitivas decidindo palíndromo por construção. Lembrando:

Uma string é palíndromo se e somente se LIFO == FIFO

Esse é o teorema, o resto é adorno. Eu sei que Bash não foi feito pra isso, mas Bash também não foi feito pra rodar a Rinha de Backend e aqui a gente está, de novo.

Se gostou dessa aventura, recomendo ler o pai dela, "You don't need Kafka".

Stay tuned, e bebam água.

Disclosure

Este artigo foi escrito usando uma ferramenta TUI que criei em Rust chamada leandronsp/devTUI, com o apoio do Claude Opus 4.7 no refinamento, correção ortográfica e contexto.

Toda a identidade do artigo é 100% minha, eu não coloco IA pra escrever por mim. LLM serve apenas como um par que navega e dá feedback em tempo real, feature também incluída no DevTUI.

There's a pattern that repeats across the entire history of computing: we take something fundamentally non-deterministic and engineer enough layers on top of it until it behaves deterministically. Logic gates did it. Artificial neural networks (ANNs) replicated it at some level. LLMs are the next frontier, and the hardest one yet.

In this article, we'll explore how engineering is crucial to addressing non-determinism, and analyze the current state of the art for LLMs.

Agenda

• Part I - The lie inside a logic gate

  • Noise margins
  • The CPU clock

• Part II - Aspirina: teaching non-determinism to behave

  • Where the chaos lives
  • Composition does the rest
  • The cost compared

• Part III - LLMs and the "Unresolved Problem"

  • Tests as noise margins
  • The agentic loop as clock
  • The type system and compiler
  • What's missing: timing analysis
  • How to mitigate that?

• Conclusion

Part I - The lie inside a logic gate

A logic gate is taught in school as a simple binary machine. You give it 0s and 1s, you get 0s and 1s back.

AND, OR and XOR are example of logic gates. They are clean, mathematical, deterministic. And that's a useful fiction.

At the physical level, a gate is a transistor (or a few of them) through which electrons flow. Electrons don't care about your Boolean algebra. The voltage on a wire is a continuous, analog value that fluctuates due to thermal noise, manufacturing variance, electromagnetic interference, and even cosmic radiation flipping bits in memory.

Fun fact: in May 2003, in Belgium, an electronic voting machine gave a candidate exactly 4096 extra votes. More than she could possibly have received. After investigation, the error was attributed to the spontaneous creation of a 13th bit in the memory of the computer. The leading explanation: a cosmic ray flipped a single bit, turning a 0 into a 1 at position 2^12, adding exactly 4096 to the candidate count.

Ok, but how does the industry get determinism out of this mess?

Noise margins

Engineers define voltage thresholds: anything below 0.8V is a 0, anything above 2.0V is a 1. The zone between those values is declared "forbidden". The circuit is designed to never operate stably there. This isn't physics; it's an engineering contract. (More on noise margins)

The CPU clock

A CPU clock isn't just a metronome. It's also a sampling strategy. You let the signal propagate through the gate (which takes time, called propagation delay), and only read the value at a specific moment: the clock edge. By then, the signal has had time to cross the threshold and stabilise. Timing is calculated to guarantee this.

But when timing fails, things may get weird. If a signal is sampled while it's still in the forbidden zone, a flip-flop can enter metastability, which is an unstable equilibrium where the output isn't quite 0 or 1 and can oscillate for an indeterminate time. This may cause real crashes in real systems.

Think of metastability as what happens when two clocks on a wall are unsynchronized. Each showing a slightly different time. "What time is it right now?", one may ask. The answer depends on which clock you look at, but you can't decide which one is right.

Inside the CPU, when facing a forbidden zone, the flip-flop faces the same dilemma. It can't decide if it's a 0 or a 1, which can corrupt the system state. Engineers mitigate this with synchronizer chains, but never eliminate it entirely.

It's the non-determinism of physics leaking through the engineering abstraction!

The key insight: the determinism of digital computing is not a property of nature. It's an engineering achievement. It's paid with noise margins, timing analysis and careful design.

Yes, it's all about engineering.

Part II - Aspirina: teaching non-determinism to behave

Aspirina is my personal project that builds a complete CPU entirely from artificial neural networks (ANNs) trained to behave as logic gates, written in Rust.

Actually I created the very first version almost 10 years ago in Elixir, when I was learning about ANNs and Elixir. Recently I decided to write a Rust version while I was learning Rust.

The inversion from Part I brings the beauty: instead of physical electrons that need to be tamed into bits, here we have ANNs - systems whose weights are initialised randomly and whose outputs are continuous floating-point numbers - that need to be tamed into 0 and 1 too.

The non-determinism here is introduced deliberately, and then engineered away. Bear with me.

Where the chaos lives - unleash the madness

A neural network starts with random weights. Its output for any input is whatever the math happens to produce. Not 0, not 1, but something like 0.6312. In case you missed, I already have an article in my blog about neural networks.

Backpropagation is the engineering response. It's the algorithm that adjusts the weights iteratively, measuring how wrong the output is (the loss) and leading weights in the direction that reduces the error. Run this for 10,000 epochs - 10,000 passes over the training data - and the network converges to weights that produce 0.002 for inputs that should be 0, and 0.997 for inputs that should be 1.

The sigmoid function (a.k.a the logistic curve) does in Aspirina what noise margins do in hardware: it squashes any continuous value into the range 0..1. It doesn't give you exact binary outputs, but it pushes values toward the extremes.

And that's exactly what we need

Combined with a threshold decision - below 0.5 we could consider 0, above 0.5 we consider 1 -, we get in Aspirina the same effect: a circuit that behaves deterministically even though its internals are continuous and learned.

Composition does the rest

Once gates are trained, they're composed to:

XOR + AND => Half Adder => Full Adder => 4-bit ALU => Memory => Registers => CPU => Assembler => Interpreter

Each layer treats the layer below as if it were perfectly deterministic, which is exactly the abstraction hierarchy of real hardware.

The non-determinism was contained at the lowest level (training), and every layer above it benefits from the illusion of determinism.

Yes, determinism is an illusion. Deal with it.

The cost compared

Compute time during training. Just as chip fabrication invests energy upfront to create reliable silicon, Aspirina invests 10,000 epochs upfront to create reliable logic. After that, inference is cheap and stable.

Now, enter LLMs.

Part III - LLMs and the "Unresolved Problem"

A large language model (LLM) is non-deterministic in a deeper and more stubborn way than either of the above.

Its weights are the result of a stochastic training process over vast data. Its output is sampled probabilistically (even with temperature=0), and there's pseudo-randomness baked in. And crucially, unlike a neural network trained on a clean truth table, an LLM's "correct output" for most inputs is fundamentally ambiguous.

Yet we're building agentic systems (Claude Code, Codex, Cursor etc) that use LLMs to write, run and iterate on code autonomously. How do we tame this?

The pattern repeats, but the engineering is particularly messier on LLM's.

Tests as noise margins

A test suite could behave as a binary verdict on the LLM's output: green or red. It doesn't matter if the model generated three different valid implementations in three runs: if all tests pass, they're equivalent from the system's perspective. The non-determinism of the model is contained below the threshold of "passes tests". This works, but only as well as the tests themselves. And unlike voltage thresholds, test coverage is always incomplete.

The agentic loop as clock

An agentic system like Claude Code doesn't generate once and deliver. It reads state (files, compiler output, test results), acts, observes the new state, and repeats. This feedback loop is structurally similar to the fetch-decode-execute cycle of a CPU or the iterative of Aspirina. Each iteration constrains the space of valid next actions. Errors are observable, and the agent can correct.

The type system and compiler as low-level validators

In a Rust project like Aspirina, the borrow checker rejects invalid outputs before tests even run. The LLM can generate wrong code and hallucinate, the compiler then refuses it, the agent observes the error message, and iterates.

This is a lower-level noise margin: just a formal filter beneath the test layer and type system.

And that's why I think that languages with strict compilers like Rust will thrive in the agentic era. Let's see.

What's missing: the equivalent of timing analysis

In hardware, engineers can prove that a circuit will work at a given clock frequency. They run static timing analysis and guarantee that every signal stabilises before every clock edge. There's no equivalent for agentic LLM systems.

We don't know how many iterations an agent will need. We can't bound the convergence time. We can't guarantee termination. An agent can loop indefinitely, or enter what I'd call "agentic metastability": making and undoing the same change repeatedly, unable to reach a stable state.

Anyone who has used Claude Code or Codex for long enough has seen this - the model oscillates between two approaches, each creating a problem that motivates reverting to the other.

Despite we can retry, add guardrails, skills, and set token budgets, in the end, we can't prove termination the way a hardware engineer proves timing closure on metastability

How to mitigate that?

In hardware, the fix for metastability is design. In agentic systems, the analog fix is clearer context, more granular tests, explicit checkpoints and tighter feedback loops; which can be described with the following practical list of skills I created and have been using on a daily basis for every project in production:

• PO (product owner): agent receives a prompt and outputs a well-scoped task based on initial requirements and prior knowledge on the codebase. Clearer context.

• TDD (test-driven development): the developer agent outputs the code, guided by TDD - red, green, refactor. Granular tests as noise margins.

• Review: agent and human checks before merging. Explicit checkpoint.

• Small PRs: small PR's enable development and testing with reviewable increments. Tighter feedback loops.

Yes, that's software engineering practices all the way down (and they are not new). Yet we don't have the formal tools to prove these fixes work. It's totally empirical.

Conclusion

At least for me, what's interesting is that each level's "solved" status depends on bounded, well-defined tasks. Logic gates are deterministic for Boolean logic. Aspirina's networks converge reliably because truth tables are finite and exact. LLMs work well on narrow, testable tasks with earlier and good feedback.

The frontier is: what does it take to engineer reliable agentic behaviour on open-ended, ambiguous, long-horizon tasks? The answer probably looks like what came before: more layers of verification, better sampling strategies (the clock), formal specifications (the truth table), and tools that can analyse convergence before execution. We just don't have those yet.

Non-determinism doesn't go away. It never has. We just keep finding better ways to push it where it can't hurt us. From voltage thresholds to synchronizer chains to temperature tuning. Every generation of computing has faced the same enemy and responded with the same weapon: engineering. Besides hype, LLMs are no different. We're just earlier in the cycle.

References

https://en.wikipedia.org/wiki/Logic_gate https://en.wikipedia.org/wiki/Electronic_voting_in_Belgium https://en.wikipedia.org/wiki/Noise_margin https://en.wikipedia.org/wiki/Metastability_(electronics) https://en.wikipedia.org/wiki/Sigmoid_function https://en.wikipedia.org/wiki/Large_language_model https://en.wikipedia.org/wiki/Backpropagation https://en.wikipedia.org/wiki/Static_timing_analysis https://github.com/leandronsp/aspirina https://github.com/leandronsp/morphine https://leandronsp.com/articles/ai-ruby-an-introduction-to-neural-networks-23f3

• Recursion is an obscure topic or you want to understand it a bit better;

• Tail call and TCO are alien communication methods and;

• Trampoline is a medicine name

Then this article is for you.

Here, I'll explain what these terms are in a didactic way and the problems they solve, with examples in Ruby. But don't worry because the examples are quite simple to understand, especially since the concepts shown here are language-agnostic.

So, come with me on this endless journey.

✋ To continue, go back to the beginning

Note: This is an English translation of the original article in Portuguese: Entendendo fundamentos de recursão

Agenda

• What is recursion

• Meet Fibo

• Tail call

• Stack and stack overflow

• Tail call optimization

• Trampoline

• Conclusion

• References

What is recursion

In computer programs, we're used to breaking large problems into smaller problems through the use of functions or methods.

Recursion is, in an extremely simplified way, a technique in computing where these problems are broken down so that a certain function is executed recursively.

With this, the function "calls itself" to solve some computation and continue its execution.

Meet Fibo

A very classic example of recursion is discovering, given the Fibonacci sequence, or Fibo, which number is found at a certain position.

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55.........

With this in place, the fib function would return results like:

fib(0) = 0
fib(1) = 1
fib(2) = 1
...
fib(7) = 13
fib(10) = 55

We then have a possible recursive implementation in Ruby:

def fib(position)
  return position if position < 2

  fib(position - 1) + fib(position - 2)
end

This code, however, is not performant. When trying to find the number at position 10_000 (ten thousand) in the sequence, the program becomes very slow because it makes numerous redundant recursive calls.

                 fib(10)
             /                \
     fib(9)                 fib(8)
        /          \          /   \
fib(8)     fib(7)     fib(7)    fib(6)
  /      \       /       \       /   \
fib(7) fib(6) fib(6) fib(5) fib(6) fib(5)
   /    \     /     \     /     \     /    \
fib(6) fib(5) fib(5) fib(4) fib(5) fib(4) fib(5) fib(4)
  /   \   /   \   /   \   /   \   /   \   /   \   /   \
...

Consequently, the larger the function input, the execution time of this code tends to grow exponentially, which in Big-O notation would be O(2^n).

Is it possible to reduce this complexity?

What if we try to apply a technique where the last function call, instead of being the sum of two recursive calls, becomes just one recursive call, without performing additional computations?

This technique exists and is called tail call, or tail recursion.

Tail call

Tail call, or TC, consists of a recursive function where the last recursive call is the function itself without additional computation.

With this in place, we reduce the complexity from exponential to linear, as if it were a simple loop iterating over a list of inputs.

In Big-O notation this becomes O(n), meaning the complexity grows linearly following the growth of the input.

Example in Ruby:

def fib(position, _current = 0, _next = 1)
  return _current if position < 1

  fib(position - 1, _next, _current + _next)
end

Therefore, the number of recursive calls is drastically reduced to something like:

fib(10, 0, 1)
fib(9, 1, 1)
fib(8, 1, 2)
fib(7, 2, 3)
fib(6, 3, 5)
fib(5, 5, 8)
fib(4, 8, 13)
fib(3, 13, 21)
fib(2, 21, 34)
fib(1, 34, 55)
fib(0, 55, 89)

Notice how the number of recursive calls decreased, meaning the code is following a more linear path with this approach.

Thus, when running the fib with TC program, the execution time is exponentially less than running without TC, being tens of thousands of times faster.

✋ Clearly a program that takes exponential time is terribly poor performance-wise, right?

# Without TC
fib(30) # 0.75 seconds

# With TC
fib(30) # 0.000075 seconds

Going back to the example of fib(10000), when running with TC, we see that execution is much faster, however:

recursion/fib.rb:10:in `fib_tc': stack level too deep (SystemStackError)
        from recursion/fib.rb:10:in `fib_tc'
        from recursion/fib.rb:10:in `fib_tc'
        from recursion/fib.rb:10:in `fib_tc'
        from recursion/fib.rb:10:in `fib_tc'
        from recursion/fib.rb:10:in `fib_tc'
        from recursion/fib.rb:10:in `fib_tc'
        from recursion/fib.rb:10:in `fib_tc'
        from recursion/fib.rb:10:in `fib_tc'

Uh oh, a stack overflow!

To understand what's happening, let's first understand what the heck is a stack and stack overflow.

Stack and stack overflow

When a program is executed, a data structure in the form of a stack, called Stack (duh), is allocated in memory and is used to store the data being used in a running function.

✋ There's also another structure in the program's memory called Heap, which is not a stack and has other traits that are beyond the scope of this article. To understand recursion, we focus only on the stack

When the program enters a function or method, each piece of data is pushed onto the stack. When the function finishes, the removal (pop) of each piece of data is done.

With each function call, a new stack frame is assigned. Since a recursive call never ends, the runtime doesn't know it needs to "pop" the data and finish the frame, so at every call, a new stack frame is created and more elements are added to the stack.

Guess what happens when we add too much data to the stack to the point of exceeding its limit in the computer's memory?

Yes, the infamous Stack overflow happens 💥🪲, and this explains that error in Ruby when running fib of 10000 with tail call.

✋ So does that mean calculating fib of 10000 is an impossible problem to solve with recursion?

Hold on, some languages employ an optimization technique that consists of using the tail call with just one stack frame, hence ensuring that each recursive call is treated as if it were an iteration in a primitive loop.

With this, the function's arguments and data are manipulated in a single stack frame, exactly as if we had written a primitive loop. And consequently, new tail recursive calls won't cause stack overflow.

We call this technique Tail call optimization, or TCO.

Tail call optimization

Due to its imperative nature, and like several other general-purpose languages, Ruby doesn't have native TCO support.

Usually this optimization is more commonly found in languages with a strong inclination toward the functional paradigm, rather than the imperative one.

But in Ruby it's possible to enable TCO mode with a simple configuration in the Ruby runtime instruction (YARV), and thus we can execute fib of 10000 without pain.

RubyVM::InstructionSequence.compile_option = {
  tailcall_optimization: true,
  trace_instruction: false
}

def fib(position, _current = 0, _next = 1)
  return _current if position < 1

  fib(position - 1, _next, _current + _next)
end

# TC with TCO
fib(10000) # 0.02 seconds

Superb! With TCO enabled, a fib 10000 with tail call is executed in 0.02 seconds!

It's worth remembering that TCO is a technique used not only in recursion but also in instruction generation optimization in compilers, but this is beyond the scope of this article.

✋ Okay, but what if it's not possible to enable TCO for tail recursion or I'm programming in a language that doesn't have TCO support?

Trampoline to the rescue.

Trampoline

To understand trampoline, let's think about the problem and a possible solution.

If we think smart, we can initially conclude that recursion should be avoided, and this is premise number one.

def fib(position, _current = 0, _next = 1)
  return _current if position < 1

  ###################################
  #### We must avoid this!!!!!! ####
  ###################################
  fib(position - 1, _next, _current + _next)
end

Premise two, instead of returning a recursive call directly, what if we return it encapsulated in an anonymous function structure that stores context to be executed in another context?

Yes, like a closure or lambda for the more attentive readers

In Ruby, we can use the concept of lambdas.

def fib(position, _current = 0, _next = 1)
  return _current if position < 1

  lambda do
    fib(position - 1, _next, _current + _next)
  end
end

If we call result = fib(0), because of the first line's short-circuit (position < 1), the method's return is 0.

But if we call result = fib(10), the return won't be a recursive call, but rather the return will be an anonymous function (lambda).

By doing this, the method is then finished and the stack is cleared, meaning the pop of data is done from within the method.

Since lambdas store context, if we call result.call, the lambda is executed with the previous context, which can return the final number (if it enters the short-circuit) or another lambda with the new context.

And so, we loop until we have the final value, while the current return continues to be a lambda. Did you understand what we can do?

Yes, a loop!

result = fib(10000)

while result.is_a?(Proc)
  result = result.call
end

puts result

Output (a really very large number):

33644764876431783266621612005107543310302148460680063906564769974680081442166662368155595513633734025582065332680836159373734790483865268263040892463056431887354544369559827491606602099884183933864652731300088830269235673613135117579297437854413752130520504347701602264758318906527890855154366159582987279682987510631200575428783453215515103870818298969791613127856265033195487140214287532698187962046936097879900350962302291026368131493195275630227837628441540360584402572114334961180023091208287046088923962328835461505776583271252546093591128203925285393434620904245248929403901706233888991085841065183173360437470737908552631764325733993712871937587746897479926305837065742830161637408969178426378624212835258112820516370298089332099905707920064367426202389783111470054074998459250360633560933883831923386783056136435351892133279732908133732642652633989763922723407882928177953580570993691049175470808931841056146322338217465637321248226383092103297701648054726243842374862411453093812206564914032751086643394517512161526545361333111314042436854805106765843493523836959653428071768775328348234345557366719731392746273629108210679280784718035329131176778924659089938635459327894523777674406192240337638674004021330343297496902028328145933418826817683893072003634795623117103101291953169794607632737589253530772552375943788434504067715555779056450443016640119462580972216729758615026968443146952034614932291105970676243268515992834709891284706740862008587135016260312071903172086094081298321581077282076353186624611278245537208532365305775956430072517744315051539600905168603220349163222640885248852433158051534849622434848299380905070483482449327453732624567755879089187190803662058009594743150052402532709746995318770724376825907419939632265984147498193609285223945039707165443156421328157688908058783183404917434556270520223564846495196112460268313970975069382648706613264507665074611512677522748621598642530711298441182622661057163515069260029861704945425047491378115154139941550671256271197133252763631939606902895650288268608362241082050562430701794976171121233066073310059947366875

🔑 Key point And with this, friends, we have the trampoline technique: a non-recursive primitive loop that keeps calling another function written recursively but that returns a lambda with context, until reaching the final value.

This code, without TCO, for fib of 10000, takes 0.04 seconds, a result very close to TCO and without causing stack overflow.

Incredible, right? Now there are no excuses for not writing a function recursively in languages that don't have TCO support 😛

Conclusion

In this article, the intent was to bring some concepts and fundamentals around the recursion topic. These concepts overlap with very academic topics that sometimes make it difficult for people who are starting in the field or who don't have a very academic background to understand.

I hope I've clarified the recursion subject in a didactic way. If you can, leave any corrections or relevant information in the comments.

References

https://en.wikipedia.org/wiki/Fibonacci_sequence https://en.wikipedia.org/wiki/Recursion https://www.geeksforgeeks.org/stack-data-structure/ https://en.wikipedia.org/wiki/Tail_call https://en.wikipedia.org/wiki/Trampoline_(computing) https://nithinbekal.com/posts/ruby-tco/ https://www.bigocheatsheet.com/ https://ruby-doc.org/core-3.1.0/RubyVM/InstructionSequence.html#method-c-compile_option

Recently I wrote a 6-article series about x86 Assembly (written in Portuguese, but I'm planning to translate the guide to English soon), covering fundamental concepts of computer architecture and low-level programming while building a minimalist multi-threaded web server.

During the process, I left some important concepts aside for later articles, because if I had tackled them during the series, it would have been even longer than it already was. However, these are concepts that can be addressed separately, like the queues implemented in the thread pool.

And when we talk about queues, it's inevitable to address arrays and how they're organized in computer memory.

In this article, we'll cover fundamental concepts like memory manipulation, registers, and heap memory while implementing arrays.

I'm assuming you're already familiar with x86 Assembly and the GDB tool. If not, I strongly recommend reading my series.

Agenda

• Arrays don't exist

• Strings don't exist either

• The simplest array in the universe

• Using an array with uninitialized data

  • Index to the rescue
  • Hitting the array limit

• Heap, heap, hooray!

  • Dynamic memory allocation with brk
  • Pointers, pointers everywhere
  • Resize with brk

• The final program

• Conclusion

• References

Arrays don't exist

Arrays don't exist. Simple as that.

As we saw in part IV of the series, memory is organized contiguously, where information is allocated one after another.

Suppose we want to declare the following sequence of information:

1, 2, 'H', 0

I know, I know, the types are mixed, but that doesn't matter right now. They all fit in 1 byte

In x86 assembly (let's call it asm for the rest of the article), we can declare this information in the data section like this:

section .data
stuff: db 0x1, 0x2, 0x48, 0x0

Remember that the character 'H' in the ASCII table represents 0x48 in hexadecimal

Using gdb for debugging, we can confirm that this hexadecimal sequence at the stuff label is stored as follows:

# Reading the first hexbyte in stuff
(gdb) x/1xb (void*) &stuff
0x402000 <stuff>:       0x01

# Reading the second hexbyte in stuff
(gdb) x/1xb (void*) &stuff+1
0x402001:       0x02

# Reading the third hexbyte in stuff
(gdb) x/1xb (void*) &stuff+2
0x402002:       0x48

We can also represent the hexadecimal 0x48 in string format using x/s:

(gdb) x/s (void*) &stuff+2
0x402002:       "H"

It's all hexadecimal!

With this, if we want to represent the string "Hello", according to the ASCII table, it could look like this:

section .data
msg: db 0x48, 0x65, 0x6C, 0x6C, 0x6F, 0x0

In gdb, let's check the string representation of the msg label:

(gdb) x/s &msg
0x402000 <msg>: "Hello"

In asm, it's possible to declare the string with direct ASCII table representation:

section .data
msg: db "Hello", 0x0

; same as
; msg: db 0x48, 0x65, 0x6C, 0x6C, 0x6F, 0x0

Strings don't exist either

In other words, it's all hexadecimal in memory. An array, like a string, is simply a contiguous sequence of data with the same size in memory.

The difference is that a string is a "special array" that has data representing ASCII table characters (note that both need to delimit a "final" byte to represent the end of the string or array):

The simplest array possible

Below we have the implementation of a very simple array in asm, which we'll explore step by step in subsequent sections:

global _start

%define SYS_exit 60
%define EXIT_SUCCESS 0

section .data
array: db 1, 2, 3, 0

section .text
_start:
	mov al, [array]        ; array[0]
	mov bl, [array + 1]    ; array[1]
	mov cl, [array + 2]    ; array[2]
	mov sil, [array + 3]   ; array[3]
.exit:
	mov rdi, EXIT_SUCCESS
	mov rax, SYS_exit
	syscall

In the initialized data section .data, we declare an array with 3 elements of 1 byte each (integers from 1 to 3), using the number 0 as the array terminator:

section .data
array: db 1, 2, 3, 0

Next, in the .text section, which is where the program's source code goes, we can access array elements using pointer arithmetic, storing the result in registers:

section .text
_start:
mov al, [array]        ; array[0]

In the code above, we're accessing the value contained at memory address 0x402000 and storing the result in a register (AL) that has a size of 1 byte, meaning only the first byte of the array will be stored in the register.

Let's check with gdb:

# The array is stored at address 0x402000
# and contains the hex value 0x00 0x03 0x02 0x01,
# remembering that this architecture uses little-endian format
(gdb) x &array
0x402000 <array>:       0x00030201

(gdb) b 13
(gdb) run
(gdb) next

# In register AL we have the first element of the array
(gdb) i r al
al             0x1                 1

# It's the same as accessing the first hexbyte contained at address
# 0x402000
(gdb) x/1xb 0x402000
0x402000 <array>:       0x01

Remember that the AL register represents the lower 8-bits within the spectrum of the RAX register which encompasses a total of 64-bits in the x86_64 architecture

To access the other array elements, just do pointer arithmetic and store in other 1-byte registers:

mov al, [array]        ; array[0] => 1
mov bl, [array + 1]    ; array[1] => 2
mov cl, [array + 2]    ; array[2] => 3
mov sil, [array + 3]   ; array[3] => 0 (array ends here)

Using an array with uninitialized data

So far, we're declaring the array in the .data section where data is initialized. But we can make the program more "dynamic" by declaring the array in the section of uninitialized data, which is .bss.

Keeping compatibility with the previous example, let's declare a 4-byte array using the resb directive which means "reserve byte", where the first 3 bytes are reserved to store array elements and the last byte representing 0x0 which is the array terminator.

section .bss
array: resb 4 ; 3 bytes + 1 terminator byte

In gdb, we can see that the array is initialized with all values at zero, indicating that the array is empty but has 4 bytes reserved:

(gdb) x &array
0x402004 <array>:       0x00000000

(gdb) x/4xb &array
0x402004 <array>:       0x00    0x00    0x00    0x00

To add elements to the array, we also need to use pointer arithmetic, just like we did in the previous example to access an array with pre-initialized data.

; Move value 1 to the first byte of the memory address at array
mov byte [array], 1  ; array[0] = 1

With gdb we confirm that at address 0x402000 where the array is, byte 1 was added:

(gdb) x &array
0x402000 <array>:       0x00000001

And if we want to add value 2 to the next byte of the array?

mov byte [array + 1], 2

(gdb) x &array
0x402000 <array>:       0x00000201

Notice that what changes is the array "index". At the initial position of the array, it's like the index is zero, and at the subsequent position, we use index 1, which can be incremented until the array terminator.

It would be very complicated to keep manipulating a hard-coded index. We need a pointer to represent this index.

Index to the rescue

Assuming the array pointer starts with zero, which is the memory address where the array is, we can declare it in the initialized data section .data:

section .bss
array: resb 4 ; 3 bytes + 1 terminator byte

section .data
pointer: db 0

So, we could add the first element like this, right?

mov byte [array + pointer], 1   ; array + 0

When running the program, we get this error:

src/live.asm:14: error: invalid effective address: multiple base segments

This error indicates that we're trying to do pointer manipulation from multiple memory segments, in this case array and pointer.

To solve this, we need to do pointer manipulation with immediate values (which was the previous case with hard-coded numbers) or with registers:

; append(1)
mov al, byte [pointer]
mov byte [array + rax], 1   ; array + 0

• the first instruction moves the first byte contained at the pointer address and stores it in register AL

• the second instruction moves immediate value 1 (array element) to the array's memory address. Since RAX (64-bit version of AL) has value 0x0 representing the pointer, we're inserting at the first byte of the array

And to store the second element in the array?

; append(2)
mov al, byte [pointer]
mov byte [array + rax], 2

In gdb, let's check what's happening:

(gdb) x &array
0x402004 <array>:       0x00000002

Uh, oh... This way we're overwriting the previous value. We actually want the pointer to "move", that is, it needs to be incremented by one byte so that append(2) results in 2 elements in the array.

With the INC instruction we can solve this problem:

mov al, byte [pointer]      ; pointer -> 0
mov byte [array + rax], 1   ; array + 0
inc byte [pointer]          ; pointer -> 1

mov al, byte [pointer]
mov byte [array + rax], 2   ; array + 1

(gdb) x &array
0x402004 <array>:       0x00000201

Yay! What a wonderful day!

Hitting the array limit

And if we keep incrementing the pointer until we hit the array limit?

mov al, byte [pointer]
mov byte [array + rax], 1   ; array + 0
inc byte [pointer]

mov al, byte [pointer]
mov byte [array + rax], 2   ; array + 1
inc byte [pointer]

mov al, byte [pointer]
mov byte [array + rax], 3   ; array + 2
inc byte [pointer]

# Reading the first 4 hexabytes of the array, we have the representation
# of the full array with all spaces occupied, remembering that
# the last byte is the array terminator
(gdb) x /4xb &array
0x402004 <array>:       0x01    0x02    0x03    0x00

# The pointer is at the end of the array
(gdb) x &pointer
0x402000 <pointer>:     0x03

Wonderful, and if we add one more element, should our program allow it?

mov al, byte [pointer]
mov byte [array + rax], 4   ; array + 3
inc byte [pointer]

# We shouldn't allow another element to be added,
# since our array was already full
(gdb) x /4xb &array
0x402004 <array>:       0x01    0x02    0x03    0x04

# The pointer is beyond the array capacity (not good...)
(gdb) x &pointer
0x402000 <pointer>:     0x04

Let's use a conditional jump (I explain more about this in the series) to not allow the element to be added. With this, before appending to the array, we should check if the pointer is already at the end of the array:

cmp byte [pointer], 3   ; check if array is full
je .exit                ; jump to .exit routine if flag is raised

Here's the complete program:

global _start

%define SYS_exit 60
%define EXIT_SUCCESS 0

section .bss
array: resb 4 ; 3 bytes + 1 terminator byte

section .data
pointer: db 0

section .text
_start:
	cmp byte [pointer], 3   ; check if array is full
	je .exit

	mov al, byte [pointer]
	mov byte [array + rax], 1   ; array + 0
	inc byte [pointer]

	cmp byte [pointer], 3   ; check if array is full
	je .exit

	mov al, byte [pointer]
	mov byte [array + rax], 2   ; array + 1
	inc byte [pointer]

	cmp byte [pointer], 3   ; check if array is full
	je .exit

	mov al, byte [pointer]
	mov byte [array + rax], 3   ; array + 2
	inc byte [pointer]

	cmp byte [pointer], 3   ; check if array is full
	je .exit

	; shouldn't allow adding the fourth element,
	; since the array supports up to 3 elements. this way,
	; we'd be writing to the memory address of other
	; program data
	mov al, byte [pointer]
	mov byte [array + rax], 4   ; array + 3
	inc byte [pointer]
.exit:
	mov rdi, EXIT_SUCCESS
	mov rax, SYS_exit
	syscall

(gdb) x &pointer
0x402000 <pointer>:     0x00000003
(gdb) x &array
0x402004 <array>:       0x00030201

Perfect, let's now do a small refactoring in the code separating the append logic into a subroutine:

global _start

%define SYS_exit 60
%define EXIT_SUCCESS 0
%define CAPACITY 3

section .bss
array: resb CAPACITY + 1

section .data
pointer: db 0

section .text
_start:
	mov rdi, 1
	call .append

	mov rdi, 2
	call .append

	mov rdi, 3
	call .append

	mov rdi, 4
	call .append
.exit:
	mov rdi, EXIT_SUCCESS
	mov rax, SYS_exit
	syscall
.append:
	cmp byte [pointer], CAPACITY ; check if array is full
	je .done

	mov al, byte [pointer]
	mov byte [array + rax], dil
	inc byte [pointer]
.done:
	ret

If you want to understand more about conditional jump, routines, call, ret and flags, I suggest reading my series which has been referenced several times in this article

Running with gdb and...

(gdb) x &array
0x402004 <array>:       0x00030201

(gdb) x &pointer
0x402000 <pointer>:     0x00000003

A big Yay!

However, there may be situations where we want our array to be resized to support more elements, that is, the array size would be dynamic.

How do we add more elements beyond the initial capacity without writing to other memory areas that don't belong to the array?

Heap, heap, hooray!

Before talking about the heap, let's remember how the memory layout of a computer program works:

• the layout is represented as an area in computer memory where we have the program's lowest memory addresses toward the highest addresses at the top

• at the lowest memory addresses, we have the .text section, where we've already seen that it refers to the program itself

• then we have the data section which encompasses initialized data .data and the following section representing uninitialized data .bss

• at the highest addresses, we have the program's stack, which stores metadata such as the program name, its arguments and any program information that has a fixed size fitting within the stack, as well as function calls and their respective arguments

• the stack has a stack format and "grows downward", that is, as we add elements to the stack, it grows toward lower addresses in memory

In the "middle" of the layout, between the data section and the stack, we have a large area in memory that many end up associating as heap. In the heap, we can allocate data dynamically, unlike the static way we do in the data section.

To accommodate a dynamic-sized array that supports resizing, we have to allocate memory in this area.

In this article, we'll call this region in the middle of memory between the data section and the stack heap

Dynamic memory allocation with brk

One way to manipulate this memory area is through the brk syscall, which changes the program break, which is where the data section ends.

With brk, we can modify this program break to higher addresses, that is, allowing manipulation of memory areas that go beyond the program and data section.

The first thing we need to do is map the syscall and make the call that brings the current break address:

%define SYS_brk 12
....

section .text
_start:
; syscall to access the program break (0x403000), which is where
; the data section ends and the heap begins
mov rdi, 0
mov rax, SYS_brk
syscall
....

With gdb, let's analyze the program state:

# Breakpoint at brk syscall line
(gdb) b 18
(gdb) run

# The program start is in the .text section and begins at
# 0x401000
(gdb) x _start
0x401000 <_start>:      0x000000bf

# The pointer is in the .data section a bit higher and starts at
# 0x402000
(gdb) x &pointer
0x402000 <pointer>:     0x00000000

# The array is in the .bss section a bit higher and starts at
# 0x402004
(gdb) x &array
0x402004 <array>:       0x00000000

# Execute the brk syscall
(gdb) n

# The brk syscall stores in RAX the program break memory address,
# in this case a bit higher at 0x403000
(gdb) i r rax
rax            0x403000            4206592

• 0x401000: .text section which is where the program begins

• 0x402000: .data section where initialized data is

• 0x402004: .bss section where uninitialized data is

• 0x403000: program break, which is where the data section ends and our "heap" begins

With this, from address 0x403000 onwards is where we'll put our array elements, so the array address can use just one byte, which points to the address where the first element begins in the heap.

In the syscall we made, if the argument in RDI is zero, it means brk will return the current program break, in this case 0x403000. But we can make more brk syscalls with a different RDI argument (incremented), signaling that we're changing the program break.

From now on, in the .bss data section, we no longer need to reserve 4 bytes for the array, so only 1 byte is needed which will represent the array's memory address in the heap:

global _start

%define SYS_brk 12
%define SYS_exit 60
%define EXIT_SUCCESS 0
%define CAPACITY 3

; initially starts with 0x000000, but will later contain
; address 0x403000
section .bss
array: resb 1

section .data
pointer: db 0

section .text
_start:
	mov rdi, 0
	mov rax, SYS_brk
	syscall

	mov rdi, rax
	add rdi, CAPACITY
	mov rax, SYS_brk
	syscall

...
...

Analyzing with gdb:

# Breakpoint at first syscall
(gdb) b 18

(gdb) run

# Execute the syscall line
(gdb) n

# In RAX the syscall stores the program break address, in this case
# 0x403000
(gdb) i r rax
rax            0x403000            4206592

(gdb) x 0x403000
0x403000:       Cannot access memory at address 0x403000

At this moment, this address is not yet accessible because we haven't reserved new bytes in the heap. Let's move forward with the next syscall:

(gdb) n
(gdb) n

# Before executing the syscall, we verify that argument RDI will
# contain the desired address for the new program break, in this case with
# 3 bytes added, 0x403003
(gdb) i r rdi
rdi            0x403003            4206595

# Execute the syscall...
(gdb) n
(gdb) n

# After executing the second syscall, we see that in RAX, the program break was changed to 0x403003
(gdb) i r rax
rax            0x403003            4206595

Now, we can access memory addresses between 0x403000 and 0x403003:

(gdb) x 0x403000
0x403000:       0x00000000
(gdb) x 0x403001
0x403001:       0x00000000
(gdb) x 0x403002
0x403002:       0x00000000
(gdb) x 0x403003
0x403003:       0x00000000

Wow! Now we have in the heap a reserved area especially for our dear array, how cool is that!

How are we going to manipulate the array in this memory region?

Pointers, pointers everywhere

After the first syscall, we should take the memory address 0x403000 which represents the first program break and store it in the array pointer that's in .bss:

...
mov rdi, 0
mov rax, SYS_brk
syscall
mov [array], rax      ; <---- breakpoint here

mov rdi, rax
add rdi, CAPACITY
mov rax, SYS_brk
syscall
...

Let's check with gdb the breakpoint at the line that changes the array pointer:

(gdb) b 19
(gdb) run

(gdb) x &array
0x402004 <array>:       0x00000000

# Execute the line that changes the pointer
(gdb) n

# Now the pointer points to address 0x403000,
# this is what we want
(gdb) x &array
0x402004 <array>:       0x00403000

Important to note that array is at address 0x402004, in the .bss section, so its value represents another memory address 0x403000 which is where the first array element should start in the heap.

Now, when we make the next syscall to allocate 3 bytes in the heap, the program break will be modified and we'll be able to manipulate the array since the pointer already points to the correct address.

After the second syscall, we can no longer manipulate array by its value, because now the array value is no longer an actual element, but rather an address to another place in memory.

Here's the program in its current version:

global _start

%define SYS_brk 12
%define SYS_exit 60
%define EXIT_SUCCESS 0
%define CAPACITY 3

section .bss
array: resb 1   ; 0x403000

section .data
pointer: db 0

section .text
_start:
	mov rdi, 0
	mov rax, SYS_brk
	syscall
	mov [array], rax

	mov rdi, rax
	add rdi, CAPACITY
	mov rax, SYS_brk
	syscall

	mov rbx, [array]

	mov r8, 1
	call .append

	mov r8, 2
	call .append

	mov r8, 3
	call .append

	mov r8, 4
	call .append
.exit:
	mov rdi, EXIT_SUCCESS
	mov rax, SYS_exit
	syscall
.append:
	cmp byte [pointer], CAPACITY ; check if array is full
	je .done

	mov sil, byte [pointer]
	mov byte [rbx + rsi], r8b
	inc byte [pointer]
.done:
	ret

Explaining each block:

mov rdi, 0
mov rax, SYS_brk
syscall
mov [array], rax

• fetches the current program break and stores the address in the array pointer

mov rdi, rax
add rdi, CAPACITY
mov rax, SYS_brk
syscall

• modifies the current program break, incrementing 3 bytes which is the initial array capacity in the heap

; assign to the register the memory address that the
; "array" pointer is pointing to
mov rbx, [array]

• stores the pointer's memory address in register RBX. This is necessary because we don't want to do arithmetic directly on the pointer in the .bss section, but rather through a register that allows it

mov r8, 1
call .append

• since now RDI was used as argument in the brk syscall, it's not convenient to use this register anymore to represent the element to be added to the array, so we switch to register R8

.append:
	cmp byte [pointer], CAPACITY ; check if array is full
	je .done

	mov sil, byte [pointer]
	mov byte [rbx + rsi], r8b    ; indirect-mode addressing
	inc byte [pointer]
.done:
	ret

Now the .append routine has been modified so that heap array manipulation is through register RBX. We also can't use register RAX anymore to represent the pointer because the brk syscall also used it as return for the program break; in this case we switch to RSI (which has SIL as its lower 8-bit representation).

Running with gdb, we can verify that elements are being added at address 0x403000 which is in the heap, through the pointer that was stored in register RBX:

# Array points to address 0x403000
(gdb) x &array
0x402004 <array>:       0x00403000

# At that address, we have the added elements. Yay!
(gdb) x 0x403000
0x403000:       0x00030201

# And the "index" pointer correctly representing the end of the array in the heap
(gdb) x &pointer
0x402000 <pointer>:     0x00000003

At this point, the program has the same behavior as the previous example with static array in .bss, not allowing adding more elements when the array reaches its limit.

Let's change this by resizing the array and allowing new elements to be added.

Resize with brk

Next, we start the steps so that array resizing is done when it reaches capacity limit. We start by changing the .append routine:

.append:
	cmp byte [pointer], CAPACITY ; check if array is full
	je .resize

	mov sil, byte [pointer]
	mov byte [rbx + rsi], r8b
	inc byte [pointer]
.done:
	ret
.resize:
	...

Instead of jumping to .done when the array is full, we jump to another subroutine called .resize, which should make the brk syscall again, thus modifying the program break in a new memory area, obeying the initial array capacity:

.append:
	cmp byte [pointer], CAPACITY ; check if array is full
	je .resize

	mov sil, byte [pointer]
	mov byte [rbx + rsi], r8b
	inc byte [pointer]
.done:
	ret
.resize:
	mov rdi, 0
	mov rax, SYS_brk
	syscall

	mov rdi, rax            ; RDI now represents the current break
	add rdi, CAPACITY       ; add 3 bytes, becoming 0x403006
	mov rax, SYS_brk
	syscall
	jmp .append

• the first resize syscall brings the current break, in this case we already know it's 0x403003, which was allocated at the beginning of the program for the array

• the second resize syscall modifies the current break, thus allocating 3 more bytes in the heap

• at the end of resize, instead of returning the function, we'll go back to the beginning of .append and execute the necessary logic to add the element to the array

This way, we can manipulate this new memory area to add more elements to the array, thus dynamically modifying its capacity.

If we run the program exactly like this, we'll face a problem, because:

• every time resize is done, it jumps to the beginning of the routine

• the array size (pointer) is checked against the initial capacity, which in this case is 3. Since the pointer reached value 3, it will enter resize again characterizing an infinite loop with infinite resize until memory runs out

To solve this, we need to compare the pointer with the current capacity (modified), and therefore we'll add a value in the .data section representing the current capacity:

%define CAPACITY 3

section .data
pointer: db 0
currentCapacity: db CAPACITY ; starts with 3

In the .append routine, we'll make the comparison with currentCapacity, which will be modified with each resize, instead of with CAPACITY, which will remain fixed with the initial value throughout the program.

.append:
	mov r9, [currentCapacity]
	cmp byte [pointer], r9b     ; check if array is full
	je .resize
...

And, after resizing before going back to .append, we'll increment the initial capacity to the current capacity:

.resize:
	mov rdi, 0
	mov rax, SYS_brk
	syscall

	mov rdi, rax
	add rdi, CAPACITY
	mov rax, SYS_brk
	syscall

	mov r10, currentCapacity
	add byte [r10], CAPACITY
	jmp .append

Running the program, we can see that element 4 was successfully added to the array after resizing:

(gdb) x 0x403000
0x403000:       0x04030201

And if we add more and more elements?

...
	mov r8, 4
	call .append

	mov r8, 5
	call .append

	mov r8, 6
	call .append

	mov r8, 7
	call .append
...

# We can see that currentCapacity is 9, meaning there were
# 2 resizes. Our array can now accommodate up to 9 elements,
# so when adding the tenth element, one more resize would be done.
(gdb) x &currentCapacity
0x402001 <currentCapacity>:     0x09

# Fetching the first 9 hexabytes at the array address in the heap
(gdb) x/9xb  0x403000
0x403000:       0x01    0x02    0x03    0x04    0x05    0x06    0x07    0x00
0x403008:       0x00

How cool is that?

The final program

Here's the final program, with an array of initial capacity of 3 elements in the heap that can be resized using the brk syscall, as more elements are added to the array:

global _start

%define SYS_brk 12
%define SYS_exit 60
%define EXIT_SUCCESS 0
%define CAPACITY 3

section .bss
array: resb 1

section .data
pointer: db 0
currentCapacity: db CAPACITY ; initial capacity is 3

section .text
_start:
	mov rdi, 0
	mov rax, SYS_brk
	syscall
	mov [array], rax

	mov rdi, rax
	add rdi, CAPACITY
	mov rax, SYS_brk
	syscall

	mov rbx, [array]

	mov r8, 1
	call .append

	mov r8, 2
	call .append

	mov r8, 3
	call .append

	mov r8, 4
	call .append

	mov r8, 5
	call .append

	mov r8, 6
	call .append

	mov r8, 7
	call .append
.exit:
	mov rdi, EXIT_SUCCESS
	mov rax, SYS_exit
	syscall
.append:
	mov r9, [currentCapacity]
	cmp byte [pointer], r9b ; check if array is full
	je .resize

	mov sil, byte [pointer]
	mov byte [rbx + rsi], r8b
	inc byte [pointer]
.done:
	ret
.resize:
	mov rdi, 0
	mov rax, SYS_brk
	syscall

	mov rdi, rax
	add rdi, CAPACITY
	mov rax, SYS_brk
	syscall

	mov r10, currentCapacity
	add byte [r10], CAPACITY
	jmp .append

Conclusion

In this article, we showed the implementation of an array in x86 Assembly, covering important concepts like memory layout, register manipulation, and dynamic memory allocation with brk.

This article is the foundation for future articles about data structures, where I intend to write about implementing queues and later other data structures.

References

• brk syscall

• x86 Assembly series

If you want to learn about UNIX signals, binary operations the easy way, how a message broker works under the hood, and a bit of Ruby, this post is for you.

And if you came here just because of the clickbait title, I apologize and invite you to keep reading. It'll be fun, I promise.

It's all about UNIX

A few days ago, I saw some discussion on the internet about how we could send messages between processes. Many people think of sockets, which are the most common way to send messages, even allowing communication across different machines and networks. Some don't even realize that pipes are another way to send messages between processes:

$ echo 'hello' | base64
aGVsbG8K

Here's what's happening:

• The process echo is started with the content "hello"

• echo is a program that prints the message to STDOUT

• Through the pipe, the content in STDOUT is sent directly to the STDIN of the base64 process

• The base64 process encodes its input to Base64 and then puts the result in STDOUT

Note the word "send". Yes, anonymous pipes are a form of IPC (Inter-process communication). Other forms of IPC in UNIX include:

• named pipes (mkfifo)

• sockets

• regular files

• or even a simple signal

UNIX signals

According to Wikipedia:

A UNIX signal is a standardized message sent to a program to trigger specific behaviour, such as quitting or error handling

There are many signals we can send to a process, including:

• SIGTERM - sends a notification to the process to terminate. It can be "trapped," which means the process can do some cleanup work before termination, like releasing OS resources and closing file descriptors

• SIGKILL - sends a termination signal that cannot be trapped or ignored, forcing immediate termination

• SIGINT - the interrupt signal, typically sent when you press Ctrl+C in the terminal. It can be trapped, allowing the process to perform cleanup before exiting gracefully

• SIGHUP - the hangup signal, originally sent when a terminal connection was lost. Modern applications often use it to reload configuration files without restarting the process

• SIGQUIT - similar to SIGINT but also generates a core dump for debugging

• SIGSTOP - pauses (suspends) a process. Cannot be trapped or ignored

• SIGCONT - resumes a process that was paused by SIGSTOP

• SIGCHLD - sent to a parent process when a child process terminates or stops

• SIGUSR1 and SIGUSR2 - user-defined signals that applications can use for custom purposes

Sending messages using signals

Okay, we know that signals are a primitive form of IPC. UNIX-like systems provide a syscall called kill that sends signals to processes. Historically, this syscall was created solely to terminate processes. But over time, they needed to accommodate other types of signals, so they reused the same syscall for different purposes.

For instance, let's create a simple Ruby script sleeper.rb which sleeps for 60 seconds, nothing more:

puts "Process ID: #{Process.pid}"
puts "Sleeping for 60 seconds..."
sleep 60

After running we see:

Process ID: 55402
Sleeping for 60 seconds...

In another window, we can send the SIGTERM signal to the process 55402 via syscall kill:

$ kill -SIGTERM 55402

And then, in the script session:

[1]    55402 terminated  ruby sleeper.rb

Signal traps

In Ruby, we can also trap a signal using the trap method in Ruby:

puts "Process ID: #{Process.pid}"
puts "Sleeping for 60 seconds..."

trap('SIGTERM') do 
  puts "Received SIGTERM, exiting gracefully..."
  exit
end

sleep 60

Which in turn, after sending the signal, will gracefully:

Process ID: 55536
Sleeping for 60 seconds...
Received SIGTERM, exiting gracefully...

After all, we cannot send messages using signals. They are a primitive way of sending standardized messages which will trigger specific behaviours. At most, we can trap some signals, but nothing more.

Okay Leandro, but what's the purpose of this article then?

Hold on. That's exactly why I'm here. To prove points by doing useless stuff, like when I simulated OOP in Bash a couple of years ago (it was fun though).

To understand how we can "hack" UNIX signals and send messages between processes, let's first talk a bit about binary operations. Yes, those "zeros" and "ones" you were scared of when you saw them for the first time. But they don't bite (🥁 LOL), I promise.

What is a message?

If we model a message as a sequence of characters, we could say that at a high-level, messages are simply strings. But in memory, they are stored as bytes.

We know that bytes are made of bits. In computer terms, what's a bit? It's simply an abstraction representing only two states:

• zero

• one

That's it. For instance, using ASCII, we know that the letter "h" has the following codes:

• 104 in decimal

• 0x68 in hexadecimal

• 01101000 in binary

Binary-wise, what if we represented each "0" with a specific signal and each "1" with another? We know that some signals such as SIGTERM, SIGINT, and SIGCONT can be trapped, but intercepting them would harm their original purpose.

But thankfully, UNIX provides two user-defined signals that are perfect for our hacking experiment.

Sending SIGUSR1 and SIGUSR2

First things first, let's trap those signals in the code:

puts "Process ID: #{Process.pid}"
puts "Sleeping forever. Send signals to this process to see how it responds."

trap('SIGUSR1') do 
  puts "Received SIGUSR1 signal"
end

trap('SIGUSR2') do
  puts "Received SIGUSR2 signal"
end

sleep

Process ID: 56172
Sleeping forever. Send signals to this process to see how it responds.

After sending some kill -SIGUSR1 56172 and kill -SIGUSR2 56172, we can see that the process prints the following content:

Process ID: 56172
Sleeping forever. Send signals to this process to see how it responds.
Received SIGUSR1 signal
Received SIGUSR2 signal
Received SIGUSR2 signal
Received SIGUSR1 signal
Received SIGUSR1 signal
Received SIGUSR2 signal

Signals don't carry data. But the example we have is perfect for changing to bits, uh?

Received SIGUSR1 signal # 0
Received SIGUSR2 signal # 1
Received SIGUSR2 signal # 1
Received SIGUSR1 signal # 0
Received SIGUSR2 signal # 1
Received SIGUSR1 signal # 0
Received SIGUSR1 signal # 0
Received SIGUSR1 signal # 0

That's exactly 01101000, the binary representation of the letter "h". We're simply encoding the letter as a binary representation and sending it via signals

Again, we're encoding it as a binary and sending it via signals.

How cool is that?

Decoding the binary data

On the other side, the receiver should be capable of decoding the message and converting it back to the letter "h":

• sender encodes the message

• receiver decodes the message

So, how do we decode 01101000 (the letter "h" in ASCII)? Let's break it down into a few steps:

1. First, we need to see the 8 bits as individual digits in their respective positions
2. The rightmost bit is at position 0, whereas the leftmost bit is at position 7. This is how we define the most significant bit (MSB, the leftmost) and the least significant bit (LSB, the rightmost)
3. For this example, we perform a left shift operation on each bit and then sum all the values, in this case from MSB to LSB (the order doesn't matter much for now): (0 << 7) + (1 << 6) + (1 << 5) + (0 << 4) + ... + (0 << 0): left shift on zeros will always produce a zero

• 0 << 7 = (2 ** 7) * 0 = 128 * 0 = 0

• 1 << 6 = (2 ** 6) * 1 = 64 * 1 = 64

Similarly to the remaining bits:

• 1 << 5 = 32

• 0 << 4 = 0

• 1 << 3 = 8

• 0 << 2 = 0

• 0 << 1 = 0

• 0 << 0 = 0

So, our sum becomes, from MSB to LSB:

MSB                          LSB
0   1    1    0   1   0   0   0
0 + 64 + 32 + 0 + 8 + 0 + 0 + 0 = 104

104 is exactly the decimal representation of the letter "h" in ASCII.

How wonderful is that?

Sending the letter "h"

Now let's convert these operations to Ruby code. We'll write a simple program receiver.rb that receives signals in order from LSB to MSB (positions 0 to 7) and then converts them back to ASCII characters, printing to STDOUT.

Basically, we'll accumulate bits and whenever we form a complete byte, we'll decode it to its ASCII representation. The very basic implementation of our accumulate_bit(bit) method would look like as follows:

@position = 0 # start with the LSB
@accumulator = 0

def accumulate_bit(bit)
  # The left shift operator (<<) is used to 
  # shift the bits of the number to the left.
  #
  # This is equivalent of: (2 ** @position) * bit
  @accumulator += (bit << @position)
  return @accumulator if @position == 7 # stop accumulating after 8 bits (byte)

  @position += 1 # move to the next bit position: 0 becomes 1, 1 becomes 2, etc.
end

# Letter "h" in binary is 01101000
# But we'll send from the LSB to the MSB
#
# 0110 1000 (MSB -> LSB) becomes 0001 0110 (LSB -> MSB)
# The order doesn't matter that much, it'll depend on 
# the receiver's implementation.
accumulate_bit(0)
accumulate_bit(0)
accumulate_bit(0)
accumulate_bit(1)
accumulate_bit(0)
accumulate_bit(1)
accumulate_bit(1)
accumulate_bit(0)

puts @accumulator # should print 104, which is the ASCII code for "h"

Pay attention to this code. It's very important and builds the foundation for the next steps. If you didn't get it, go back and read it again. Try it yourself in the terminal or using your preferred programming language.

Now, how to convert the decimal 104 to the ASCII character representation? Luckily, Ruby provides a method called chr which does the job:

irb> puts 104.chr
=> "h"

We could do the same job for the rest of the word "hello", for instance. According to the ASCII table, it should be the following:

• e in decimal is 101

• l in decimal is 108

• o in decimal is 111

Let's check if Ruby knows that:

104.chr    # "h"
101.chr    # "e"
108.chr    # "l"
111.chr    # "o"

We can even "decode" the word to the decimal representation in ASCII:

irb> "hello".bytes
=> [104, 101, 108, 108, 111]

Now, time to finish our receiver implementation to properly print the letter "h":

@position = 0 # start with the LSB
@accumulator = 0

trap('SIGUSR1') { decode_signal(0) }
trap('SIGUSR2') { decode_signal(1) }

def decode_signal(bit)
  accumulate_bit(bit)
  return unless @position == 8 # if not yet accumulated a byte, keep accumulating

  print "Received byte: #{@accumulator} (#{@accumulator.chr})\n"

  @accumulator = 0 # reset the accumulator
  @position = 0 # reset position for the next byte
end

def accumulate_bit(bit)
  # The left shift operator (<<) is used to 
  # shift the bits of the number to the left.
  #
  # This is equivalent of: (2 ** @position) * bit
  @accumulator += (bit << @position)
  @position += 1 # move to the next bit position: 0 becomes 1, 1 becomes 2, etc.
end

puts "Process ID: #{Process.pid}"
sleep

Read that code and its comments. It's very important. Do not continue reading until you really get what's happening here.

• Whenever we get SIGUSR1, we accumulate the bit 0

• When getting SIGUSR2, accumulate then the bit 1

• When accumulator reaches the position8, it means we have a byte. At this moment we should print the ASCII representation using the .chr we seen earlier. Then, reset bit position and accumulator

Let's see our receiver in action! Start the receiver in one terminal:

$ ruby receiver.rb
Process ID: 58219

Great! Now the receiver is listening for signals. In another terminal, let's manually send signals to form the letter "h" (which is 01101000 in binary, remember?):

  # Sending from LSB to MSB: 0, 0, 0, 1, 0, 1, 1, 0
  $ kill -SIGUSR1 58219  # 0
  $ kill -SIGUSR1 58219  # 0
  $ kill -SIGUSR1 58219  # 0
  $ kill -SIGUSR2 58219  # 1
  $ kill -SIGUSR1 58219  # 0
  $ kill -SIGUSR2 58219  # 1
  $ kill -SIGUSR2 58219  # 1
  $ kill -SIGUSR1 58219  # 0

And in the receiver terminal, we should see:

Received byte: 104 (h)

How amazing is that? We just sent the letter "h" using only two UNIX signals!

But wait. Manually sending 8 signals for each character? That's tedious and error-prone. What if we wanted to send the word "hello"? That's 5 characters × 8 bits = 40 signals to send manually. No way.

We need a sender.

Building the sender

The sender's job is the opposite of the receiver: it should encode a message (string) into bits and send them as signals to the receiver process.

Let's think about what we need:

1. Take a message as input (like "hello")
2. Convert each character to its byte representation
3. Extract the 8 bits from each byte
4. Send SIGUSR1 for bit 0, SIGUSR2 for bit 1
5. Repeat for all characters

The tricky part here is the step 3: how do we extract individual bits from a byte? To extract the bit at position i, we can use the following formula:

bit = (byte >> i) & 1

Let me break this down:

• byte >> i performs a right shift by i positions

• & 1 is a bitwise AND operation that extracts only the rightmost bit

For the letter "h" (01101000 in binary, 104 in decimal):

Position 0 (LSB):

• (104 >> 0) = 104 / (2 ** 0) = 104 / 1 = 104

• 01101000 >> 0 = 01101000

• 01101000 & 00000001 = 0 (one AND zero is zero)

Position 1:

• (104 >> 1) = 104 / (2 ** 1) = 104 / 2 = 52

• 01101000 >> 1 = 00110100

• 00110100 & 00000001 = 0

Position 2:

• (104 >> 2) = 104 / (2 ** 2) = 104 / 4 = 26

• 01101000 >> 2 = 00011010

• 00011010 & 00000001 = 0

Position 3:

• (104 >> 3) = 104 / (2 ** 3) = 104 / 8 = 13

• 01101000 >> 3 = 00001101

• 00001101 & 00000001 = 1 (one AND one equals one)

And so on for positions 4, 5, 6, and 7. This gives us: 0, 0, 0, 1, 0, 1, 1, 0 — exactly the bits we need from LSB to MSB!

• (104 >> 0) & 1 = 104 & 1 = 0

• (104 >> 1) & 1 = 52 & 1 = 0

• (104 >> 2) & 1 = 26 & 1 = 0

• (104 >> 3) & 1 = 13 & 1 = 1

• (104 >> 4) & 1 = 6 & 1 = 0

• (104 >> 5) & 1 = 3 & 1 = 1

• (104 >> 6) & 1 = 1 & 1 = 1

• (104 >> 7) & 1 = 0 & 1 = 0

Pay close attention to this technique. It's a fundamental operation in low-level programming.

So now time to build the sender.rb which is pretty simple:

receiver_pid = ARGV[0].to_i
message = ARGV[1..-1].join(' ')

def encode_byte(byte)
  8.times.map do |i|
    # Extract each bit from the byte, starting from the LSB
    (byte >> i) & 1
  end
end

message.bytes.each do |byte|
  encode_byte(byte).each do |bit|
    signal = bit == 0 ? 'SIGUSR1' : 'SIGUSR2'
    Process.kill(signal, receiver_pid)
    sleep 0.001 # Delay to allow the receiver to process the signal
  end
end

For each byte (8-bit structure) we extract the bit performing the right shift + AND oprerations. The result is the extracted bit.

In the receiver window:

$ ruby receiver.rb
Process ID: 68968

And in the sender window:

$ ruby sender.rb 68968 h

The receiver will print:

$ ruby receiver.rb
Process ID: 68968
Received byte: 104 (h)

Processes sending messages with only two signals! How wonderful is that?

Sending the "hello" message

Now, sending the hello message is super easy. The sender is already able to send not only a letter but any message using signals:

$ ruby sender.rb 68968 hello

# And the receiver:
Received byte: 104 (h)
Received byte: 101 (e)
Received byte: 108 (l)
Received byte: 108 (l)
Received byte: 111 (o)

Just change the receiver implementation a little bit:

def decode_signal(bit)
  accumulate_bit(bit)
  return unless @position == 8 # if not yet accumulated a byte, keep accumulating

  print @accumulator.chr # print the byte as a character

  @accumulator = 0 # reset the accumulator
  @position = 0 # reset position for the next byte
end

And then:

$ ruby sender.rb 96875 Hello

# In the receiver's terminal
Process ID: 96875
Hello

However, if we send the message again, the receiver will print everything in the same line:

$ ruby sender.rb 96875 Hello
$ ruby sender.rb 96875 Hello

# In the receiver's terminal
Process ID: 96875
HelloHello

It's obvious: the receiver doesn't know where the sender finished the message, so it's impossible to know where we should stop one message and print the next one on a new line with \n.

We should then determine how the sender indicates the end of the message. How about being it all zeroes (0000 0000)?

• We send the message: first 5 bytes representing the "hello" message

• Then we send a "NULL terminator", just one byte 0 (0000 0000)

0110 1000 # h
0110 0101 # e
0110 1000 # l
0110 1000 # l
0110 1111 # o
0000 0000 # NULL

Hence, when the receiver gets a NULL terminator, it will print a line feed \n. Let's change the sender.rb first:

receiver_pid = ARGV[0].to_i
message = ARGV[1..-1].join(' ')

def encode_byte(byte)
  8.times.map do |i|
    # Extract each bit from the byte, starting from the LSB
    (byte >> i) & 1
  end
end

message.bytes.each do |byte|
  encode_byte(byte).each do |bit|
    signal = bit == 0 ? 'SIGUSR1' : 'SIGUSR2'
    Process.kill(signal, receiver_pid)
    sleep 0.001 # Delay to allow the receiver to process the signal
  end
end

# Send NULL terminator (0000 0000)
8.times do
  Process.kill('SIGUSR1', receiver_pid)
  sleep 0.001 # Delay to allow the receiver to process the signal
end

puts "Message sent to receiver (PID: #{receiver_pid})"

Then, the receiver.rb:

@position = 0 # start with the LSB
@accumulator = 0

trap('SIGUSR1') { decode_signal(0) }
trap('SIGUSR2') { decode_signal(1) }

def decode_signal(bit)
  accumulate_bit(bit)
  return unless @position == 8 # if not yet accumulated a byte, keep accumulating

  if @accumulator.zero? # NULL terminator received
    print "\n"
  else
    print @accumulator.chr # print the byte as a character
  end

  @accumulator = 0 # reset the accumulator
  @position = 0 # reset position for the next byte
end

def accumulate_bit(bit)
  # The left shift operator (<<) is used to 
  # shift the bits of the number to the left.
  #
  # This is equivalent of: (2 ** @position) * bit
  @accumulator += (bit << @position)
  @position += 1 # move to the next bit position: 0 becomes 1, 1 becomes 2, etc.
end

puts "Process ID: #{Process.pid}"
sleep

Output:

$ ruby sender.rb 96875 Hello, World!
$ ruby sender.rb 96875 You're welcome
$ ruby sender.rb 96875 How are you?

# Receiver
Process ID: 97176
Hello, World!
You're welcome
How are you?

OMG Leandro! That's amazing!

Amazing, right? We just built an entire communication system between two processes using one of the most primitive methods available: UNIX signals.

The sky's the limit now! Why not build a full-fledged message broker using this crazy technique?

A modest message broker using UNIX signals

We'll break down the development into three components:

1. Broker: the intermediary that routes messages
2. Consumer: processes that receive messages
3. Producer: processes that send messages

1. Let's start with the Broker. It should register itself with the producer, then trap incoming signals, decode them, and enqueue the messages for delivery to consumers via outgoing signals:

#!/usr/bin/env ruby

require_relative 'signal_codec'
require_relative 'consumer'

class Broker 
  PID = 'broker.pid'.freeze

  def initialize
    @codec = SignalCodec.new
    @queue = Queue.new
    @consumer_index = 0
  end

  def start 
    register_broker

    trap('SIGUSR1') { process_bit(0) }
    trap('SIGUSR2') { process_bit(1) }
    
    puts "Broker PID: #{Process.pid}"
    puts "Waiting for messages..."

    distribute_messages

    sleep # Keep alive
  end 

  private

  def process_bit(bit)
    @codec.accumulate_bit(bit) do |message|
      @queue.push(message) unless message.empty?
    end
  end

  def register_broker 
    File.write(PID, Process.pid)
    at_exit { File.delete(PID) if File.exist?(PID) }
  end

  def distribute_messages
    Thread.new do
      loop do
        sleep 0.1

        next if @queue.empty?

        consumers = File.exist?(Consumer::FILE) ? File.readlines(Consumer::FILE).map(&:to_i) : []
        next if consumers.empty?

        message = @queue.pop(true) rescue next

        consumer_pid = consumers[@consumer_index % consumers.size]
        @consumer_index += 1

        puts "[SEND] #{message} → Consumer #{consumer_pid}"

        @codec.send_message(message, consumer_pid)
      end
    end
  end
end

if __FILE__ == $0 
  broker = Broker.new
  broker.start
end

• The broker registers itself

• Traps incoming signals USR1 (bit 0) and USR2 (bit 1)

• Enqueues the messages

• Send messages to consumers using outgoing signals (USR1 and USR2 too)

Note that we're using a module called SignalCodec which will be explained soon. Basically this module contains all core components to encode/decode signals and perform bitwise operations.

2. Now the Consumer implementation:

#!/usr/bin/env ruby

require_relative 'signal_codec'

class Consumer
  FILE = 'consumers.txt'.freeze

  def initialize
    @codec = SignalCodec.new
  end

  def start
    register_consumer

    trap('SIGUSR1') { process_bit(0) }
    trap('SIGUSR2') { process_bit(1) }

    puts "Consumer PID: #{Process.pid}"
    puts "Waiting for messages..."

    sleep # Keep alive
  end

  private

  def process_bit(bit)
    @codec.accumulate_bit(bit) do |message|
      puts "[RECEIVE] #{message}"
    end
  end

  def register_consumer
    File.open(FILE, 'a') { |f| f.puts Process.pid }
    at_exit { deregister_consumer }
  end

  def deregister_consumer
    if File.exist?(FILE)
      consumers = File.readlines(FILE).map(&:strip).reject { |pid| pid.to_i == Process.pid }
      File.write(FILE, consumers.join("\n"))
    end
  end
end

if __FILE__ == $0
  consumer = Consumer.new
  consumer.start
end

• The consumer starts and registers itself with the broker

• Consumer then traps incoming signals (bit 0 and bit 1)

• Decodes and prints messages

3. Last but not least, the Producer implementation, which is pretty straightforward:

#!/usr/bin/env ruby

require_relative 'signal_codec'
require_relative 'broker'

unless File.exist?(Broker::PID)
  abort "Error: Broker not running (#{Broker::PID} not found)"
end

broker_pid = File.read(Broker::PID).strip.to_i
message = ARGV.join(' ')

if message.empty?
  puts "Usage: ruby producer.rb <message>"
  exit 1
end

codec = SignalCodec.new

puts "Sending: #{message}"
codec.send_message(message, broker_pid)
puts "Message sent to broker (PID: #{broker_pid})"

• Producer receives a ASCII message from the STDIN

• Encode and sends the message to the broker via outgoing signals

So far, this architecture should look familiar. Many broker implementations follow these basic foundations.

Of course, production-ready implementations are far more robust than this one. Here, we're just poking around with hacking and experimentation

The coolest part is the SignalCodec though:

class SignalCodec 
  SIGNAL_DELAY = 0.001 # Delay between signals to allow processing

  def initialize
    @accumulator = 0
    @position = 0
    @buffer = []
  end

  def accumulate_bit(bit)
    @accumulator += (bit << @position)
    @position += 1

    if @position == 8 # Byte is complete
      if @accumulator.zero? # Message complete - NULL terminator
        decoded = @buffer.pack("C*").force_encoding('UTF-8')
        yield(decoded) if block_given?
        @buffer.clear
      else 
        @buffer << @accumulator
      end

      @position = 0
      @accumulator = 0
    end
  end

  def send_message(message, pid)
    message.each_byte do |byte|
      8.times do |i|
        bit = (byte >> i) & 1
        signal = bit == 0 ? 'SIGUSR1' : 'SIGUSR2'
        Process.kill(signal, pid)
        sleep SIGNAL_DELAY
      end
    end

    # Send NULL terminator (0000 0000)
    8.times do
      Process.kill('SIGUSR1', pid)
      sleep SIGNAL_DELAY
    end
  end
end

If you've been following along, this shouldn't be hard to understand, but I'll break down how this beautiful piece of code works:

• The codec is initialized with the bit position at zero, as well as the accumulator

• A buffer is also initialized to store accumulated bits until a complete byte is formed

• The accumulate_bit method should be familiar from our earlier implementation, but it now accepts a closure (block) that lets the caller decide what to do with each decoded byte

• send_message encodes a message into bits and sends them via UNIX signals

Everything in action:

How cool, amazing, wonderful, impressive, astonishing is that?

Conclusion

Yes, we built a message broker using nothing but UNIX signals and a bit of Ruby magic. Sure, it's not production-ready, and you definitely shouldn't use this in your next startup (please don't), but that was never the point.

The real takeaway here isn't the broker itself: it's understanding how the fundamentals work. We explored binary operations, UNIX signals, and IPC in a hands-on way that most people never bother with.

We took something "useless" and made it work, just for fun. So next time someone asks you about message brokers, you can casually mention that you once built (or saw) one using just two signals. And if they look at you weird, well, that's their problem. Now go build something equally useless and amazing. The world needs more hackers who experiment just for the fun of it.

Happy hacking!

Sentado no sofá e assistindo Frozen, tive a ideia de escrever sobre minha retrospectiva deste ano. Nunca fiz isso antes, então bora lá, porque acho que foi muita coisa.

Mas antes, um aftermath de 2023

2023 foi um ano bastante agitado pra mim. Passei por uma tireoidectomia (mas estou bem, obrigado) e também foi o ano em que resolvi fazer "learn in public" e deixar tudo gravado no meu canal do Youtube.

Iniciei cobrindo a rinha de compiladores, onde submeti uma versão em Ruby, e depois fui trazendo conteúdo para iniciantes em Rust. Teve também transmissão ao vivo criando uma Rede Neural Artificial em Ruby, então vi que eu realmente estava gostando de compartilhar minha jornada coding in public.

Na parte de artigos, escrevi muita coisa em 2023:

• Introdução ao Tekton CI/CD

• Kubernetes 101

• Um guia completo cobrindo os fundamentos de Git

• Criando redes neurais em Ruby

• Teve até artigo sobre ponto flutuante

• Fundamentos de recursão

• Resumo da rinha de compiladores e trampolim

• Mais Rust

E o famoso Guia Web 101 também.

Mentira, esse guia web foi em 2021, mas eu quis colocar ele aqui só pra fazer propaganda mesmo

As metas para 2024

No fim de 2023 estabeleci algumas metas pra 2024 nessa parte de criação de conteúdo. Mas não foram metas muito arrojadas pois eu queria dar uma desaquecida do que foi o agitado 2023.

Dentre as metas estava continuar explorando Rust; escrever um guia completo de concorrência; criar um interpretador em Ruby; fazer lives com Kubernetes; falar sobre tédio e; Awk.

Sim, Awk

Podemos confirmar isto em meio a tantos rascunhos que tenho nesta plataforma:

E chegou 2024

Comecei o ano de 2024 focado em me aprofundar em Rust, tanto que em Janeiro até cheguei a criar o crust (outro CRUD, mas desta vez em Rust) e o aspirina (outra rede neural, mas desta vez em Rust) durante algumas lives. Queria também iniciar meus estudos em Rust na parte de I/O assíncrono. Era esse o plano.

Era.

A Rinha de Backend, 2ª edição

Mas aí veio a rinha de backend do Zan pra tirar meu foco e resolvi fazer lives compartilhando minha solução submetida, que inicialmente seria em Ruby, com o agostinho.

Ledo engano. Acabei por submeter 5 diferentes soluções:

• 3 em Ruby: agostinho, que usa meus micro frameworks favoritos Adelnor e Chespirito; tortuga que não usa framework nem biblioteca alguma, o puro suco de uma linguagem criada no Japão; e tonico, uma versão sem frameworks que usa I/O assíncrono all the way down

• 1 em Rust: quokka, criado durante uma live

• e claro, a famosa versão em Bash, canabrava

Foram muitas horas em lives, inclusive esta onde eu mostrava como criar uma thread pool e connection pool em Rust.

Nossa Leandro, como você arranja tempo pra fazer lives? Eu não consigo ter tempo pra isso

Isso é problema seu, e não meu.

Um leve sopro do Gleam e o grande tutorial de Assembly

No início do ano notei um pequeno hype em cima do Gleam. Decidi explorar em live. Até que gostei da linguagem, e estava determinado a continuar estudando.

Mas aí minha amiga e meu amigo, algumas pessoas do trabalho começaram a me provocar. Ficavam colocando Assembly na minha frente. Foi quando numa 6a feira sem pretensão, no Discord, eu e mais alguns colegas fizemos um tutorial rápido de Assembly x86.

Foi quando pensei "ta aí, vou aprender esse negócio e criar um web server multi-threaded simples em Assembly, compartilhando a jornada tanto em artigos quanto em lives".

A saga do Assembly x86

Fiz várias lives pela manhã (que eu chamava de lives matinais, duh), mostrando o desenvolvimento do web server e minha saga de aprendizado. Foi incrível, pude ter contato com pessoas como o Blau Araújo que é referência em conteúdo pt-BR em Assembly e outras coisas de baixo nível.

Tem um vídeo no meu canal, que é a minha "sincera" reação quando o server finalmente funcionou devolvendo a primeira resposta HTTP, ao vivo em live.

Não obstante, resolvi também escrever artigos em live. Me diga, quem em sã consciência acompanha alguém escrevendo um artigo em live durante umas 4 ou 5 horas?

Não sei quem é mais maluco

Brincadeiras a parte, e ideia é mesmo compartilhar o processo. É sobre o modus operandi, a forma como eu quebro o raciocínio em partes na hora de escrever. E também o que me inspira.

No mundo dos artigos, eu comecei em Abril a saga "Escrevendo um Web server em Assembly x86". Foram um total de 6 artigos, onde no final conseguimos implementar um web server simples, porém multi-threaded em Assembly. Foram 3 meses escrevendo, fazendo lives e muitos conceitos abordados, muita coisa mesmo.

Aproveitei também para escrever sobre Arrays em Assembly x86.

Esta saga foi muito enriquecedora pra mim. Pude aprender e firmar muitos conceitos. Vale muito a pena aprender Assembly e coisas de baixo nível.

Enfim o "Guia de Concorrência 101"

Quando finalizei a saga de Assembly, resolvi voltar para uma das coisas que eu tinha como meta para 2024: escrever sobre concorrência. Este é um tema que estudo há mais de 5 anos quase que diariamente, experimentando e validando conceitos.

Foi então que bem agora, agorinha mesmo (risos) no final de Novembro que comecei a escrever o guia de concorrência (pt-BR). Por enquanto já abordei conceitos de concorrência no sistema operacional e como a linguagem C implementa as principais primitivas de concorrência. Mas o intuito é cobrir com mais linguagens de programação: Ruby, Python, PHP, NodeJS, Go, Rust, Elixir, Java, Kotlin e mais o que vier à cabeça.

Com uma brincadeira no bluesky, o Rodolfo de Nadai (meu primeiro investidor) deu uma ideia de eu lançar um "buy me a coffee" neste guia. Lancei e gostei da ideia, tanto que no momento são 23 apoiadores do projeto. Apesar de que faço de forma genuína sem interesse financeiro, pois defendo muito o conhecimento livre, este apoio da galera tem sido crucial para que eu continuasse, desde o apoio com Pix, revisão ou mesmo compartlihamento do conteúdo.

Gratidão a todos vocês que fazem isto acontecer ❤

Misc

Outras coisas que explorei ao longo deste ano, enquanto ia focando nas coisas de Assembly e concorrência:

• leandronsp/necelu, brincando com Lucene em Java (relembrando the good old days)

• open telemetry: OTel é um assunto bem interessante, onde quero me aprofundar em 2025

• leandronsp/yacs, Yet Another City Search, uma busca textual ultra-rápida em PostgreSQL em mais de 12 milhões de geonames/cidades

O que esperar pra 2025

Para 2025, espero mergulhar mais fundo em Rust, explorar OpenTelemetry e, quem sabe, encarar outra linguagem inusitada.

Afinal, aprender nunca é demais. 🚀

Feliz 2025 a todes <3

E também já faz bastante tempo que tenho o interesse em voltar a escrever mais como fazia antigamente, quando tudo o que tinha era algo na mente e simplesmente convertia em um punhado de texto solto no meu blog, como essa relíquia aqui.

Gosto de escrever artigos técnicos, mas por outro lado também gosto de simplesmente sentar e começar a escrever sem grandes pretensões. Só que me faltava algo. Eu só não sabia o que era.

Para entender o que me faltava, vamos dar uns passos atrás e entender o contexto de como eu vinha utilizando plataformas de blogging e o cenário de blogging moderno.

Blogpost

A primeira plataforma que utilizei foi o Blogger, antigo blogspot, onde comecei a escrever meus primeiros artigos quando eu ainda "mexia com TI". O ano era 2009.

Eu compartilhava estudo de programação quando tinha terminado a faculdade e também as gambiarras que eu fazia instalando e corrigindo problemas no Windows XP com Service Pack 3®.

Atualmente o editor é assim, mas na época era bastante limitado:

Não é ruim até, mas repara como que o espaço não é aproveitado. E uma coisa que eu sempre prezo ao escrever é, no caso, o conforto em escrever. Gosto de uma pegada fluída, com fonte agradável e que tenha um bom contraste. No caso desta plataforma em questão, eu tenho que clicar num botão de "Preview" e assim sou direcionado para outra página, saindo completamente do contexto.

Isso me entristece, muito.

Foram bons momentos como blogueirinho no Blogger, mas fiquei uns anos sem escrever por conta de vários motivos até que...

Medium

Circa 2014 decidi experimentar voltar a escrever, mas desta vez no Medium. Fiz apenas um punhado de artigos sem grandes pretensões, mas eu tinha gostado da fluidez (na época) de escrever na plataforma deles.

Entretanto a experiência de escrever ainda tava longe de ser a que eu queria. Mas eu nem sabia o que queria.

Cadê o botão de preview? Oh, gosh...

Fast-forward para 2021.

DEV.to e Hashnode

A plataforma dev.to me pareceu a princípio bastante intuitiva. Tem um lance de comunidade que é bacana, porque para além de ser uma plataforma de blogging é também uma comunidade em volta disso. Sem contar com o suporte a Markdown que nos dias de hoje é crucial para escrever artigos.

Decidi criar uma conta lá e desde então tenho publicado quase 100 artigos:

Ao mesmo tempo, a experiência de escrever era a melhor de todas que eu encontrei, mas ainda faltava uma coisa que eu já estava começando a perceber: preview em tempo real.

É muito chato você escrever bastante coisa e ter que clicar num botão lá em cima pra abrir outra página que, não raramente, demora pra carregar. É uma experiência que não sou fã 100%.

Na mesma época eu queria algo parecido mas que pudesse ter uma "landing page" no estilo dos blogs antigos com um menu customizado com os links que eu quisesse colocar. Para além do suporte a Markdown e um editor fluído. Foi quando conheci o Hashnode.

Com Hashnode, eu pude criar uma página "customizada" com menu e assim organizar melhor meus artigos. Mas me faltava ainda algo mais configurável.

Pra não mencionar que a experiência de escrita continuava a ser a mesma: ter que clicar num botão de preview que, com uma latência absurda, faz 412 coisas menos mostrar o preview que você verdadeiramente quer naquele momento.

Acabou que deixei o Hashnode flopar, mas ficou com meu domínio leandronsp.com e passei a escrever mais pelo DEV.to mesmo.

Era o que tinha pra janta.

Mas Leandro, existem ferramentas de visualização de Markdown, você consegue usar isso no Obsidian, na linha de comando, com plugin do Vim, com extensão do VSCode, etc etc

Calma, calabreso. Eu quero apenas escrever, ver em tempo real o que estou escrevendo sendo renderizado e a seguir publicar. That's it. Pra mim o web browser é ainda a ferramenta onde sinto mais conforto ao fazer esse tipo de tarefa.

Desculpa galera do terminal, mas como um heavy-terminal user e Vimer raiz, eu ainda prefiro blogar no browser

Foi assim até 2 dias atrás.

A vontade de escrever sobre UNIX signals

Como aqueles momentos em que do nada me vem a cabeça experimentar algo inútil que ninguém pediu e que provavelmente não vai pedir, decidi criar um broker de mensageria utilizando apenas sinais UNIX.

Em breve publico isto, foi divertido

Então, como de praxe, resolvi sentar pra escrever sobre. No DEV.to como habitual. Mas quer saber?

Cansei.

Resolvi criar meu próprio motor de blog que iria me atender em todos os aspectos que sinto falta nos motores atuais: aproveitar espaço, trazer preview em tempo real, navegação fluída, simples e sem muito ruído, tudo com suporte a Markdown.

Eu queria algo mais ou menos assim:

Com isto em mente, resolvi criar o curupira.

Curupira

Pra quem já me conhece sabe que sou péssimo com nomes, mas resolvi dar esse nome em homenagem ao nosso folclore brasileiro e ao protetor das nossas matas. Só porque sim.

No intuito de ter uma navegação fluída como a que eu sonhava, decidi utilizar Phoenix LiveView. Em resumo, é uma tecnologia que escala absurdamente por conta da VM que roda por baixo, a BEAM. Meu objetivo de ter de um lado o editor para escrever e do outro um preview em tempo real poderia ser finalmente atingido de forma simples.

É muito, mas muito simples criar apps real-time com Phoenix LiveView. A seguir uma demonstração de como pode ficar a página inicial do curupira:

Basicamente você tem de um lado uma bio que resume teu blog, e do outro a lista de artigos. É isto. Não precisa ser complicado, temos que aproveitar o máximo de espaço possível na tela.

Dá também pra alternar entre tema claro e escuro:

Além de tudo isso, é possível também gerar o site estático a partir de um comando make.

Isto vai ser melhorado em breve, quero também deixar o processo de geração do site estático mais fluído e UX-friendly em um futuro próximo

leandronsp.com

Após gerar o site estático, mudei o apontamento do meu domínio principal para o Cloudflare Pages, mas poderia ser qualquer outro como Github Pages, Vercel, etc (esta parte ainda está manual no momento, vem mais coisa em breve).

Meu novo site ficou então assim:

Preferi fazer com que a página do artigo tivesse espaço suficiente para o conteúdo, e não para ruídos.

Conclusão

É isto, pra quem tiver curiosidade em ver o repositório leandronsp/curupira, publiquei hoje mas ainda está WIP.

Este artigo já foi escrito com o curupira, e em breve será publicado neste blog.

Durante o processo, acabei deixando de lado alguns conceitos importantes para artigos posteriores, pois se fosse abordar durante a saga, iria ficar maior do que já foi. Entretanto são conceitos que podem ser tratados à parte, como no caso das filas implementadas na thread pool.

E quando falamos de filas, fica inevitável abordar arrays e como estes são organizados na memória do computador.

Neste artigo, vamos abordar conceitos fundamentais como manipulação de memória, registradores e memória heap ao longo da implementação de arrays.

Vou assumir que você já tem familiaridade com Assembly x86 e a ferramenta GDB. Caso não tenha, recomendo fortemente a leitura da minha saga.

Agenda

• Arrays não existem

• Strings também não existem

• O array mais simples do universo

• Utilizando um array com dados não inicializados

  • Índice para o resgate
  • Atingindo o limite do array

• Heap, heap, hooray!

  • Alocação dinâmica de memória com brk
  • Ponteiros, ponteiros everywhere
  • Resize com brk

• O programa final

• Conclusão

• Referências

Arrays não existem

Arrays não existem. Simples assim.

Como vimos na parte IV da saga, a memória é organizada de forma contígua, onde as informações são alocadas uma após a outra.

Supondo que queremos declarar a seguinte sequência de informações:

1, 2, 'H', 0

Eu sei, eu sei, os tipos estão misturados, mas isto não importa para este momento. Todos eles cabem em 1 byte

Em assembly x86 (vamos chamar de asm no restante do artigo), podemos declarar estas informações da seguinte forma na seção de dados:

section .data
stuff: db 0x1, 0x2, 0x48, 0x0

Lembrando que o caracter 'H' na tabela ASCII representa 0x48 em hexadecimal

Ao utilizarmos o gdb para fazer debugging, podemos confirmar que esta sequência de hexadecimal no rótulo stuff está armazenada da seguinte forma:

# Leitura do primeiro hexabyte em stuff
(gdb) x/1xb (void*) &stuff
0x402000 <stuff>:       0x01

# Leitura do segundo hexabyte em stuff
(gdb) x/1xb (void*) &stuff+1
0x402001:       0x02

# Leitura do terceiro hexabyte em stuff
(gdb) x/1xb (void*) &stuff+2
0x402002:       0x48

Podemos também representar o hexadecimal 0x48 em formato de string utilizando x/s

(gdb) x/s (void*) &stuff+2
0x402002:       "H"

É tudo hexadecimal!

Com isto, caso queiramos representar a string "Hello", de acordo com a tabela ASCII, poderia ficar assim:

section .data
msg: db 0x48, 0x65, 0x6C, 0x6C, 0x6F, 0x0

No gdb vamos verificar a representação string do rótulo msg:

(gdb) x/s &msg
0x402000 <msg>: "Hello"

Em asm, é possível declarar a string já com a representação direta da tabela ASCII:

section .data
msg: db "Hello", 0x0

; é o mesmo que
; msg: db 0x48, 0x65, 0x6C, 0x6C, 0x6F, 0x0

Strings também não existem

Ou seja, é tudo hexadecimal na memória. Um array, assim como uma string, é simplesmente uma sequência contígua de dados com mesmo tamanho na memória.

A diferença é que a string é um "array especial" que tem dados que representam caracteres da tabela ASCII (note que ambos precisam delimitar um byte "final" para representar o término da string ou array):

O array mais simples possível

A seguir temos a implementação de um array bastante simples em asm, pelo que iremos explorar cada passo nas seções subsequentes:

global _start

%define SYS_exit 60
%define EXIT_SUCCESS 0

section .data
array: db 1, 2, 3, 0

section .text
_start:
	mov al, [array]        ; array[0]
	mov bl, [array + 1]    ; array[1]
	mov cl, [array + 2]    ; array[2]
	mov sil, [array + 3]   ; array[3]
.exit:
	mov rdi, EXIT_SUCCESS
	mov rax, SYS_exit
	syscall

Na seção de dados inicializados .data, declaramos um array com 3 elementos de 1 byte cada (inteiros de 1 a 3), utilizando o número 0 como término do array:

section .data
array: db 1, 2, 3, 0

A seguir, na seção .text, que é onde vai o código fonte do programa, podemos acessar os elementos do array utilizando aritmética de ponteiros, armazenando o resultado em registradores:

section .text
_start:
mov al, [array]        ; array[0]

No código acima, estamos acessando o valor contido no endereço de memória 0x402000 e armazenando o resultado em um registrador (AL) que tem o tamanho de 1 byte, ou seja, vai ser armazenado no registrador apenas o primeiro byte do array.

Vamos conferir com gdb:

# O array está armazenado no endereço 0x402000
# e contém o valor em hexa 0x00 0x03 0x02 0x01,
# lembrando que esta arquitetura utiliza o formato little-endian
(gdb) x &array
0x402000 <array>:       0x00030201

(gdb) b 13
(gdb) run
(gdb) next

# No registrador AL temos o primeiro elemento do array
(gdb) i r al
al             0x1                 1

# É o mesmo que acessar o primeiro hexabyte contido no endereço
# 0x402000
(gdb) x/1xb 0x402000
0x402000 <array>:       0x01

Lembrando que o registrador AL representa os 8-bits menores dentro do espectro do registrador RAX que contempla um total de 64-bits na arquitetura x86_64

Para acessar os demais elementos do array, basta fazer aritmética de ponteiros e ir armazenando em outros registradores de 1 byte:

mov al, [array]        ; array[0] => 1
mov bl, [array + 1]    ; array[1] => 2
mov cl, [array + 2]    ; array[2] => 3
mov sil, [array + 3]   ; array[3] => 0 (aqui termina o array)

Utilizando um array com dados não inicializados

Até agora, estamos declarando o array na seção .data onde os dados são inicializados. Mas podemos deixar o programa mais "dinâmico", declarando o array na seção de dados não inicializados, que é a .bss.

Mantendo a compatibilidade com o exemplo anterior, vamos declarar um array de 4 bytes utilizando a diretiva resb que significa "reserve byte", onde os 3 primeiros bytes são reservados para armazenar elementos do array e o último byte representando 0x0 que é o término do array.

section .bss
array: resb 4 ; 3 bytes + 1 byte de término

No gdb, podemos ver que o array está inicializado com os valores tudo a zero, o que indica que o array está vazio mas com 4 bytes reservados:

(gdb) x &array
0x402004 <array>:       0x00000000

(gdb) x/4xb &array
0x402004 <array>:       0x00    0x00    0x00    0x00

Para adicionar elementos no array, precisamos também utilizar aritmética de ponteiros, tal como fizemos no exemplo anterior para acessar um array com dados pré-inicializados.

; Move o valor 1 para o primeiro byte do endereço de memória em array
mov byte [array], 1  ; array[0] = 1

Com gdb confirmamos que no endereço 0x402000 que é onde está o array, foi adicionado o byte 1:

(gdb) x &array
0x402000 <array>:       0x00000001

E se quisermos adicionar o valor 2 no próximo byte do array?

mov byte [array + 1], 2

(gdb) x &array
0x402000 <array>:       0x00000201

Repare que o que modifica é o "índice" do array. Na posição inicial do array, é como se o índice fosse zero, e na posição subsequente, utilizamos o índice 1, podendo incrementar até o término array.

Seria muito complicado ficar manipulando índice hard-coded. Precisamos de um ponteiro para representar este índice.

Índice para o resgate

Assumindo que o ponteiro do array começa com zero, que é o endereço de memória onde está o array, podemos declará-lo na seção de dados inicializados .data:

section .bss
array: resb 4 ; 3 bytes + 1 byte de término

section .data
pointer: db 0

Logo, poderíamos adicionar o primeiro elemento da seguinte forma, certo?

mov byte [array + pointer], 1   ; array + 0

Ao rodar o programa, temos o seguinte erro:

src/live.asm:14: error: invalid effective address: multiple base segments

Este erro indica que estamos tentando fazer manipulação de ponteiros a partir de múltiplos segmentos na memória, no caso o array e pointer.

Para resolver isto, precisamos fazer manipulação de ponteiros com valores imediatos (que foi o caso anterior com número hard-coded) ou com registradores:

; append(1)
mov al, byte [pointer]
mov byte [array + rax], 1   ; array + 0

• a primeira instrução move o primeiro byte contido no endereço de pointer e armazena no registrador AL

• a segunda instrução move o valor imediato 1 (elemento do array) para o endereço de memória do array. Como em RAX (versão 64-bits de AL) temos o valor 0x0 que representa o ponteiro, então estamos fazendo a inserção no primeiro byte do array

E para armazenar o segundo elemento no array?

; append(2)
mov al, byte [pointer]
mov byte [array + rax], 2

No gdb, vamos verificar o que está acontecendo:

(gdb) x &array
0x402004 <array>:       0x00000002

Uh, oh... Desta forma estamos sobrescrevendo o valor anterior. Queremos na verdade que o ponteiro "ande", ou seja, precisa ser incrementado em um byte para que o append(2) resulte com os 2 elementos no array.

Com a instrução INC podemos resolver este problema:

mov al, byte [pointer]      ; pointer -> 0
mov byte [array + rax], 1   ; array + 0
inc byte [pointer]          ; pointer -> 1

mov al, byte [pointer]
mov byte [array + rax], 2   ; array + 1

(gdb) x &array
0x402004 <array>:       0x00000201

Yay! Que dia maravilhoso!

Atingindo o limite do array

E se continuarmos incrementando o ponteiro até atingir o limite do array?

mov al, byte [pointer]
mov byte [array + rax], 1   ; array + 0
inc byte [pointer]

mov al, byte [pointer]
mov byte [array + rax], 2   ; array + 1
inc byte [pointer]

mov al, byte [pointer]
mov byte [array + rax], 3   ; array + 2
inc byte [pointer]

# Lendo os primeiros 4 hexabytes do array, temos a representação
# do array cheio com todos os espaços ocupados, lembrando que
# o último byte é o término do array
(gdb) x /4xb &array
0x402004 <array>:       0x01    0x02    0x03    0x00

# O ponteiro está no fim do array
(gdb) x &pointer
0x402000 <pointer>:     0x03

Maravilha, e se adicionar mais um elemento, nosso programa deveria permitir?

mov al, byte [pointer]
mov byte [array + rax], 4   ; array + 3
inc byte [pointer]

# Não deveríamos permitir que mais um elemento fosse adicionado,
# pois nosso array já estava cheio
(gdb) x /4xb &array
0x402004 <array>:       0x01    0x02    0x03    0x04

# O ponteiro está para além da capacidade array (not good...)
(gdb) x &pointer
0x402000 <pointer>:     0x04

Vamos utilizar um jump condicional (explico mais sobre isto na saga) para não permitir que o elemento seja adicionado. Com isto, antes de fazer o append no array, devemos verificar se o ponteiro já não está no fim do array:

cmp byte [pointer], 3   ; verifica se o array está cheio
je .exit                ; salta para a rotina .exit caso a flag seja levantada

Assim fica o programa completo:

global _start

%define SYS_exit 60
%define EXIT_SUCCESS 0

section .bss
array: resb 4 ; 3 bytes + 1 byte de término

section .data
pointer: db 0

section .text
_start:
	cmp byte [pointer], 3   ; verifica se o array está cheio
	je .exit
	
	mov al, byte [pointer]
	mov byte [array + rax], 1   ; array + 0
	inc byte [pointer]
	
	cmp byte [pointer], 3   ; verifica se o array está cheio
	je .exit

	mov al, byte [pointer]
	mov byte [array + rax], 2   ; array + 1
	inc byte [pointer]

	cmp byte [pointer], 3   ; verifica se o array está cheio
	je .exit

	mov al, byte [pointer]
	mov byte [array + rax], 3   ; array + 2
	inc byte [pointer]

	cmp byte [pointer], 3   ; verifica se o array está cheio
	je .exit

	; não deveria permitir adicionar o quarto elemento,
	; pois o array suporta até 3 elementos. desta forma,
	; estaríamos escrevendo no endereço de memória de outros
	; dados do programa
	mov al, byte [pointer]
	mov byte [array + rax], 4   ; array + 3
	inc byte [pointer]
.exit:
	mov rdi, EXIT_SUCCESS
	mov rax, SYS_exit
	syscall

(gdb) x &pointer
0x402000 <pointer>:     0x00000003
(gdb) x &array
0x402004 <array>:       0x00030201