embeddedprep.com

Tag: FreeRTOS CAN Bus

  • Top 7 FreeRTOS Bugs That Crash Devices After 2–3 Days of Runtime

    FreeRTOS Bugs often cause devices to crash after days of uptime due to task, queue, interrupt, and memory issues. Learn real causes and proven fixes.

    FreeRTOS systems often run perfectly for hours or even days before suddenly crashing, hanging, or resetting without warning. These failures are rarely caused by obvious bugs. Instead, they come from subtle FreeRTOS issues that only appear after long runtime, real workloads, and unpredictable timing between tasks, interrupts, and memory.

    In this deep-dive article, a senior embedded systems engineer with over 10 years of real-world FreeRTOS debugging experience breaks down the top 7 FreeRTOS bugs that crash devices after 2–3 days of runtime. This is not a beginner tutorial or academic theory. Every bug discussed here is based on real production failures seen in automotive, industrial, and IoT devices.

    You will learn why common FreeRTOS bugs stay hidden during testing, how scheduling and heap behavior change over time, and why interrupt timing and task interaction are the real culprits behind long-runtime crashes. The article explains stack overflows that do not crash immediately, queue misuse involving QueueHandle_t, busy wait loops that slowly starve the scheduler, incorrect interrupt priorities, memory corruption caused by missing memory barriers, event group misuse, and CAN bus deadlocks under load.

    Each bug includes a real-world scenario, why it passes initial testing, the exact failure mechanism, how it leads to a hard fault, hang, or watchdog reset, and how both beginners and experienced developers accidentally introduce it. You will also learn practical detection techniques using FreeRTOS backtrace analysis, coredumps, GDB, Segger Embedded Studio, tracing tools, and debugger workflows that actually work in production environments.

    This article is written for real developers who debug real systems. If you are dealing with unexplained FreeRTOS crashes, long-term stability issues, or devices that fail only after days of uptime, this guide will help you recognize the patterns, confirm the root cause, and fix the problem correctly without guesswork.

    Introduction

    If you have worked with FreeRTOS long enough, you have seen this pattern.

    The device boots fine.
    Runs all tests.
    Survives stress testing.
    Customer deploys it.
    Two or three days later, it locks up, resets, or goes silent.

    Logs show nothing useful.
    Watchdog resets keep happening.
    Reboot fixes it… temporarily.

    These are not beginner mistakes like forgetting to start the scheduler. These are long-runtime FreeRTOS bugs, and they are brutal to debug.

    Why?

    Because:

    • They depend on timing
    • They depend on task interaction
    • They depend on memory behavior over time
    • They almost never show up in short tests

    I have debugged these issues on industrial controllers, automotive gateways, medical devices, and IoT hardware. The same patterns keep repeating.

    This article is for:

    • Beginners who want to avoid future pain
    • Intermediate devs stuck chasing random resets
    • Experienced engineers who want a clean freertos bug list they can cross-check against their system

    Everything here comes from real debugging sessions, real post-mortems, and real devices sitting on my desk at 3 AM.

    Why Long-Runtime Bugs Are Different in FreeRTOS

    Scheduling hides problems

    FreeRTOS is extremely forgiving at first.

    Tasks run.
    Queues pass data.
    Events fire.
    Interrupts work.

    But scheduling creates time-based behavior, not immediate behavior. A task might run fine for 10,000 iterations before it finally collides with another task at just the wrong moment.

    Heap behavior changes over time

    Even if you never call pvPortMalloc() after startup, fragmentation can still happen indirectly.

    Message buffers.
    Timers.
    Deferred interrupts.
    Drivers doing hidden allocations.

    Heap corruption rarely crashes immediately. It poisons memory slowly until something important gets overwritten.

    Interrupt timing is never stable

    What worked at low traffic fails under real load.

    CAN bursts.
    Button spam.
    GPIO storms.
    DDS or sensor interrupts firing faster than expected.

    That is why debugging FreeRTOS issues always requires thinking in timelines, not just code paths.

    Top 7 FreeRTOS Bugs That Crash Devices After 2–3 Days

    🐞 Bug #1: Stack Overflow That Does Not Crash Immediately

    Real-world scenario

    A task handles protocol parsing. It has a local buffer, some structs, maybe a JSON decode. Everything works.

    After two days, device resets with a freertos hardfault.

    Why it passes initial testing

    • Stack usage is almost enough
    • Typical test messages are smaller
    • Worst-case path is rare

    FreeRTOS does not magically know your worst execution path.

    Failure mechanism

    Stack grows slowly over time.
    Eventually it overwrites:

    • Task control block
    • Queue structures
    • Event group memory

    Now the scheduler trips over corrupted data.

    Result

    • Random reset
    • Silent hang
    • Hard fault inside the kernel

    When you finally capture it, the freertos backtrace looks meaningless.

    Beginner mistake

    Assuming default stack sizes are fine.

    Experienced mistake

    Adding “just a bit more stack” without measuring.

    How to detect it

    • Enable stack overflow checking
    • Use uxTaskGetStackHighWaterMark
    • Inspect task stacks in freertos gdb or freertos segger embedded studio

    If you ever use freertos+trace, stack usage spikes become obvious.

    How to fix it

    • Measure worst-case stack usage
    • Avoid large local arrays
    • Move buffers to static memory

    This alone eliminates a shocking number of freertos bugs.

    🐞 Bug #2: Queue Misuse with QueueHandle_t

    Real-world scenario

    You use queues everywhere. Sensor task pushes data. Processing task pulls it. Clean design.

    Two days later, tasks stop responding.

    Why it passes initial testing

    Queues hide errors extremely well.

    Sending to a deleted queue.
    Using a stale freertos queuehandle_t.
    Sending from ISR without the ISR-safe API.

    All of these may work… until timing shifts.

    Failure mechanism

    Queue internal structures get corrupted.
    Eventually scheduler walks invalid memory.
    Boom.

    Result

    • Deadlock
    • Kernel assert
    • Hard fault

    Often shows up as corrupted lists during debugging FreeRTOS.

    Beginner mistake

    Not checking return values from xQueueSend.

    Experienced mistake

    Passing queue handles across modules with no ownership rules.

    How to detect it

    • Enable asserts
    • Validate queue handles
    • Capture freertos coredump and inspect queue internals

    How to fix it

    • One owner per queue
    • Clear lifecycle rules
    • Use ISR APIs correctly

    Queues are powerful, but they are not magic.

    🐞 Bug #3: Busy Wait Instead of Proper Blocking

    Real-world scenario

    A task waits for hardware.

    So someone writes:

    while(!flag) {}

    Classic freertos busy wait.

    Why it passes initial testing

    CPU is fast.
    Load is low.
    It “works”.

    Failure mechanism

    Busy wait:

    • Starves lower priority tasks
    • Prevents idle task from running
    • Blocks memory cleanup
    • Delays timers

    Over days, timing drift builds up.

    Result

    • Watchdog resets
    • Tasks never unblock
    • System slowly degrades

    Beginner mistake

    Not understanding freertos blocking APIs.

    Experienced mistake

    Using busy wait to “optimize latency”.

    How to detect it

    • CPU usage stuck near 100%
    • Idle task not running
    • Trace shows starvation

    How to fix it

    Use:

    • vTaskDelay
    • xQueueReceive
    • freertos wait for event

    FreeRTOS is event-driven. Fight that and it fights back.

    🐞 Bug #4: Incorrect Interrupt Priorities

    Real-world scenario

    GPIO interrupt fires on button press. CAN interrupt fires on bus traffic. Everything works.

    After heavy load, system crashes.

    Why it passes initial testing

    Interrupt timing is light.
    No nesting issues.

    Failure mechanism

    Wrong priority interrupts calling FreeRTOS APIs.

    This violates kernel rules and corrupts internal state.

    Result

    • Scheduler corruption
    • Random asserts
    • freertos hardfault

    Often blamed on hardware.

    Beginner mistake

    Ignoring priority rules in interrupts FreeRTOS.

    Experienced mistake

    Porting code from another RTOS without rechecking priorities.

    How to detect it

    • Audit ISR priorities
    • Check which ISRs call RTOS APIs
    • Debug with freertos debugger

    How to fix it

    • Follow max syscall interrupt priority rules
    • Separate pure hardware ISRs from RTOS-aware ISRs

    This bug alone explains many “it crashes only under load” cases.

    🐞 Bug #5: Missing Memory Barriers

    Real-world scenario

    Shared data between ISR and task. Flags look correct. Logic is sound.

    Still fails.

    Why it passes initial testing

    Compiler optimizations are minimal.
    Timing is lucky.

    Failure mechanism

    Without a freertos memory barrier, compiler or CPU reorders memory access.

    Task sees stale data.
    ISR thinks task handled it.

    Result

    • Missed events
    • Deadlocks
    • Random behavior

    Very hard to reproduce.

    Beginner mistake

    Not understanding memory ordering.

    Experienced mistake

    Assuming volatile is enough.

    How to detect it

    • Review shared variables
    • Look for lock-free code
    • Inspect assembly in freertos gdb

    How to fix it

    • Use FreeRTOS synchronization primitives
    • Insert memory barriers where required
    • Avoid lock-free sharing unless necessary

    This is one of the sneakiest freertos bugs.

    🐞 Bug #6: Event Group Misuse

    Real-world scenario

    Multiple tasks wait on events. Button interrupt sets a flag. Processing task waits.

    Using freertos events everywhere feels elegant.

    Why it passes initial testing

    Low contention.
    Clear logic.

    Failure mechanism

    • Event bits cleared too early
    • Tasks waiting incorrectly with freertos wait for event
    • Missed signals

    Eventually tasks block forever.

    Result

    • Silent hang
    • System appears alive but does nothing

    Beginner mistake

    Using events like queues.

    Experienced mistake

    Over-clever event bit combinations.

    How to detect it

    • Log event transitions
    • Use trace tools
    • Inspect blocked task lists

    How to fix it

    • Use events only for signaling
    • Use queues for data
    • Keep event logic simple

    🐞 Bug #7: CAN Bus Deadlocks

    Real-world scenario

    CAN RX interrupt pushes frames. Processing task sends responses.

    Works fine until bus traffic spikes.

    Why it passes initial testing

    Test traffic is clean.
    No overload.

    Failure mechanism

    • RX ISR blocks indirectly
    • TX queue fills
    • Tasks wait on each other

    Classic freertos can bus deadlock.

    Result

    • Bus silence
    • Tasks blocked forever
    • Watchdog reset

    Beginner mistake

    Doing too much inside CAN ISR.

    Experienced mistake

    Ignoring backpressure handling.

    How to detect it

    • Monitor queue depth
    • Log ISR execution time
    • Inspect blocked tasks

    How to fix it

    • Minimal ISR work
    • Dedicated CAN worker task
    • Proper flow control

    Final Thoughts

    If you are chasing a crash that happens after days, it is not random.

    It is almost always:

    • Stack
    • Queues
    • Interrupts
    • Blocking
    • Memory ordering
    • Events
    • Bus contention

    Every single item in this freertos bug list has caused real devices to fail in the field.

    If you take one thing from this article, let it be this:

    FreeRTOS does exactly what you tell it to do.
    Even when that destroys your system slowly.

    Debug patiently.
    Trace behavior over time.
    Trust evidence, not assumptions.

    And next time someone suggests a freertos bug bounty, you will know exactly where to look first.

    FAQ on FreeRTOS Bugs

    1. Why do FreeRTOS devices crash after 2–3 days of runtime?

    Most long-runtime crashes are caused by hidden FreeRTOS Bugs such as slow stack overflows, queue misuse, interrupt priority mistakes, or memory corruption that only appears after repeated task execution and timing drift.

    2. What are the most common FreeRTOS Bugs in production systems?

    The most common FreeRTOS Bugs include task stack overflows, incorrect QueueHandle_t usage, busy wait loops instead of proper blocking, wrong interrupt priorities, event group misuse, and CAN bus deadlocks under load.

    3. How can a stack overflow crash a FreeRTOS system without an immediate fault?

    In FreeRTOS, a stack overflow often overwrites nearby kernel data instead of crashing instantly. The system keeps running until the scheduler or queue logic touches corrupted memory, leading to a delayed hard fault or reset.

    4. How do I detect FreeRTOS Bugs that only happen after long uptime?

    Enable stack overflow checks, use runtime stats, capture coredumps, and analyze task states with a FreeRTOS debugger. Long-runtime bugs usually require observing behavior over time, not just single execution paths.

    5. Why is busy wait dangerous in FreeRTOS?

    Busy wait loops prevent the scheduler and idle task from running properly. Over time, this starves lower-priority tasks, delays timers, increases CPU load, and eventually causes system instability or watchdog resets.

    6. What happens if interrupt priorities are wrong in FreeRTOS?

    If interrupts call FreeRTOS APIs from invalid priority levels, the kernel’s internal data structures can become corrupted. This often leads to random crashes, asserts, or hard faults that are very difficult to reproduce.

    7. Can QueueHandle_t misuse really crash a FreeRTOS system?

    Yes. Using invalid or stale QueueHandle_t values, sending to deleted queues, or using non-ISR-safe APIs inside interrupts can corrupt queue memory and eventually crash the scheduler.

    8. How do FreeRTOS event groups cause deadlocks?

    Event groups can cause deadlocks when bits are cleared too early, multiple tasks wait on the same event incorrectly, or events are used to pass data instead of just signals. This leads to tasks blocking forever.

    9. Why do CAN bus issues appear after days in FreeRTOS systems?

    Under heavy traffic, CAN receive interrupts, transmit queues, and processing tasks can block each other. Without proper flow control, FreeRTOS CAN bus handling can deadlock and silently stop communication.

    10. What tools are best for debugging FreeRTOS Bugs?

    GDB, Segger Embedded Studio, FreeRTOS trace tools, and coredump analysis are the most effective. These tools help inspect task states, backtraces, stack usage, and scheduler behavior over long runtimes.

    11. Are FreeRTOS memory barriers really necessary?

    Yes. Missing memory barriers can cause the compiler or CPU to reorder memory access between tasks and interrupts. This leads to subtle, timing-dependent bugs that appear only under real workloads.

    12. Why do FreeRTOS Bugs pass testing but fail in the field?

    Lab tests rarely reproduce real timing, interrupt frequency, bus load, or long-term memory behavior. FreeRTOS Bugs often depend on rare timing windows that only occur in real deployments.

    13. Can FreeRTOS coredumps help with long-runtime crashes?

    Absolutely. A FreeRTOS coredump allows you to inspect task stacks, queue states, and scheduler data at the moment of failure, which is critical for diagnosing crashes that happen after days.

    Recommended Resource: Expand Your ESP32 Knowledge

    If you’re enjoying this project and want to explore more powerful sensor integrations, make sure to check out my detailed guide on using the ESP32 with the DS18B20 temperature sensor. It’s a beginner-friendly, real-world tutorial that shows how to measure temperature with high accuracy and integrate the data into IoT dashboards, automation systems, or cloud servers. You can read the full step-by-step guide here: ESP with DS18b20

    This resource pairs perfectly with your ESP32 with RFID setup—together, you can build advanced smart home systems, environmental monitoring tools, or complete multi-sensor IoT projects.