No. 0391 min read
Logs are for the poor bastard awake at 3 a.m.
Tell me what failed, what might already have happened, and what I can safely do next.
No. 0451 min read
I used Splunk at Raymond James and ELK at Sprint and Jabil. Let's count the operating labor and retention bill before anybody starts worshipping a logo.
Before you pick a side in the Splunk[1]Ref. 1Splunk splunk.com versus ELK[2]Ref. 2ELK elastic.co argument, tell me who has to keep the damned thing running. And who gets the bill.
I implemented Splunk log analysis at Raymond James[3]Ref. 3Raymond James raymondjames.com and ELK at Sprint[4]Ref. 4Sprint en.wikipedia.org and Jabil[5]Ref. 5Jabil jabil.com. That doesn't make me qualified to announce which one every company on earth should buy, although apparently that has never stopped anybody from having an opinion.
Splunk is commercial software, with pricing that varies by product and agreement. Its published pricing information includes workload and ingest approaches. I'd get a quote for what we're actually going to run, because somebody will confidently repeat a price per unit they remember as though it were a law of physics.
Meanwhile, ELK means Elasticsearch[6]Ref. 6Elasticsearch elastic.co, Logstash[7]Ref. 7Logstash elastic.co and Kibana[8]Ref. 8Kibana elastic.co, and self-managing that stack does not make the expenses disappear. Compute and storage cost money. Retention costs money. Recovery testing and the people operating the cluster cost money (yes, even when those people already work here). Subscription features, support and hosted offerings need their own comparison. Calling ELK free leaves the budget unfinished before we even get to the licensing argument.
I want to compare both tools with the same data and access requirements, and count the engineering hours spent maintaining ingestion pipelines and fixing capacity problems. Leaving payroll out does make the software invoice look prettier.
For an acceptance test, I'd give an ordinary operator an unfamiliar failure and a limited time range, and see whether they can find the affected work. The vendor engineer has already practiced the demonstration. I want something our operator can investigate with, and enough budget to keep it working.
Then let the poor fuck get back to work.
No. 0391 min read
Tell me what failed, what might already have happened, and what I can safely do next.
No. 0382 min read
Cron starts commands when the clock matches, folks.
No. 0371 min read
Unix rename can publish a finished file or claim a queue item.
No. 0341 min read
Your parser rejected the record.