Log rotation in industrial traffic storage is one of those housekeeping tasks that draws almost no attention until something breaks. A poorly configured rotation policy lets log files grow unchecked, eventually filling available disk space and causing the controller software to stop recording entirely. By that point, the fault logs and compliance records that engineers rely on for diagnosis are either gone or frozen in time. Getting rotation right is straightforward once the mechanics are understood.
What log rotation actually does
Log rotation replaces a single, ever-growing log file with a managed sequence of smaller files. When a file hits a defined size or age threshold, the system closes it, renames it with a timestamp or sequence number, and opens a fresh file for new entries. Older files in the sequence are compressed or deleted according to a retention rule. The result is a predictable, bounded footprint on the storage medium.
In a standard server environment this is routine. In a roadside traffic cabinet it's more consequential. The storage volume is small, the operating environment is harsh, and the write activity is continuous. As covered in the article on data logging in traffic signal cabinets, a modern controller records phase events, detector states, fault codes, and comms activity across multiple log streams simultaneously. Each stream needs its own rotation policy, because fault logs, detector event logs, and diagnostic traces have very different volume and retention requirements.
Rotation triggers: size, time, and event-based approaches
There are three practical triggers for log rotation in traffic storage systems.
Size-based rotation closes a log file once it reaches a defined byte limit. This is the most predictable approach for high-volume streams like detector event logs, which can generate several megabytes per day at a busy intersection. A 10 MB cap per file is a common starting point, but the right figure depends on the available storage and the number of concurrent log streams.
Time-based rotation rolls the log at a fixed interval regardless of size. Daily rotation suits fault logs and signal-phase records, where correlation with calendar dates simplifies forensic review. Weekly rotation works for lower-volume streams like firmware update logs.
Event-based rotation is less common but useful in traffic deployments. A power restoration event, a controller reboot, or a comms failover can trigger an immediate rotation so that the pre-event and post-event records are in separate files. This makes incident analysis faster and reduces the risk that a single large file straddles the fault boundary.
Most deployments use a combination. Detector event logs rotate on size; fault and compliance logs rotate daily; a reboot or power event triggers an immediate rotation across all streams.
Retention depth and storage budgeting
Rotation without a retention policy just trades one problem for another. Rotating daily produces 365 files per year per stream if nothing is ever deleted. On a 4 GB industrial flash module shared across a traffic cabinet's operating system, configuration data, and log streams, that isn't sustainable.
A workable starting framework is to retain 30 days of compressed detector event logs, 90 days of fault logs, and 12 months of phase-event compliance records. Those figures aren't universal; they should be validated against the data retention policies for industrial traffic systems set by the relevant transport authority, which typically specify minimum periods for audit and incident investigation purposes.
Compression matters here. Gzip compression on text-based log files commonly achieves 70–80% size reduction. A 10 MB raw fault log becomes roughly 2–3 MB compressed. That difference compounds significantly across 90 days of retention on a capacity-constrained device.
Write endurance and rotation frequency
Log rotation involves write operations: closing a file, compressing it, writing the compressed output, deleting the original. On flash-based storage, every write cycle consumes a fraction of the medium's finite endurance budget. Frequent rotation on a high-volume stream can accelerate wear faster than the underlying log writes themselves.
This is why size-based rotation at a generous threshold (say, 50 MB rather than 1 MB) is preferable to time-based rotation every few hours for high-frequency streams. Fewer rotation events mean fewer large-block writes and a longer usable life for the storage medium. The trade-off is that individual log files are larger, which slightly complicates indexed searching. For traffic cabinet deployments, write endurance almost always wins that argument.
The interaction between rotation frequency and write endurance is one reason that write-endurance specifications deserve close attention during storage media selection. A medium rated for 3,000 program/erase cycles behaves very differently under a rotation policy generating 24 compression cycles per day versus one generating 1.
Handling rotation failures gracefully
Rotation can fail. The disk may be full before rotation completes, the compression tool may crash, or a filesystem error may prevent the rename operation. Without a failure handler, the logging daemon either stops writing or overwrites the active log indefinitely.
Traffic cabinet software should treat a rotation failure as a reportable fault, not a silent condition. The controller should log the rotation failure to a separate, minimal-size error stream (exempt from the failed rotation) and surface the condition through the remote monitoring interface. Operations staff who receive a "disk near capacity" alert during business hours can intervene before the next rotation cycle fails.
A secondary safeguard is a low-water-mark deletion policy: if free disk space drops below a defined threshold (typically 10% of partition capacity), the oldest rotated files are deleted immediately regardless of their scheduled retention period. This is a last-resort mechanism, not a substitute for a correctly sized retention policy. Its existence should be documented so that post-incident reviewers know the oldest available log may not match the nominal retention window.
Integration with remote log forwarding
Many traffic cabinet deployments forward log data to a central traffic management server or a regional data aggregator in near-real-time. In that architecture, local rotation policy can be more aggressive: once a file has been successfully forwarded, the local copy can be deleted rather than retained for the full local retention period. This keeps the on-device footprint small while preserving the full retention window at the central store.
The condition is the forwarding confirmation. Deleting a local log before confirming receipt at the remote system risks a gap in the record if the comms link was degraded during the forwarding window. A reliable forwarding agent maintains a send queue, marks files as confirmed only on receipt acknowledgement, and treats unacknowledged files as non-deletable regardless of age. Without that confirmation step, the "aggressive local rotation" approach creates exactly the record gaps it was meant to avoid.
Bob Panich Traffic Signals designs and delivers industrial data storage configurations for traffic signal and smart city deployments across Australia, including log rotation policies calibrated to the specific storage hardware, controller software, and authority-mandated retention requirements of each site.