Osquery is a powerful tool that allows you to investigate and monitor a myriad of endpoint activity, status, and configuration information through a unified SQL interface. Inside osquery, there's typically a 1:1 correspondence between a source of information and the SQL table you can use to browse or search this information. Some sources of information include parts of the /proc file system, API calls to container daemons, reading logs or status files on disk, and event streams coming from the Linux audit frame.
Between these information sources and the SQL table is a generator function, written in C++, that handles the reading and parsing of the information and populating the table. Adding a new source of information, such as supporting a new kind of application configuration, would traditionally require defining a new SQL table schema and writing the C++ code to populate it. If you're thinking that sounds like a lot of work, especially just to read a new configuration file, you're not alone. That's where Augeas fits into the pipeline.
Augeas is a remarkably generic tool that allows you to read and modify any configuration file through a unified hierarchical key/value data model. It does this by providing a large number of "lenses" that explain how to convert a configuration file into the unified model and back. If you're familiar with them, it's somewhat of a hybrid between AWK and YACC/Bison, except it works bi-directionally. If there isn't already a lens defined for your application's configuration file, you can write one yourself, and optionally contribute it back to the Augeas project for the benefit of others.
Let's quickly look at how Augeas works through its stand-alone tool called, aptly, "augtool."
Upon launching augtool, Augeas will load every config file it knows how to interpret with the lenses installed on your system. It presents it to you as a single virtual tree which you can ls or print (recursive ls) just like you would in a normal shell. Let’s say I’d like to see which package repositories my Ubuntu system will pull from when I run apt-get. I can just run:
I’ve stripped out the comment lines for clarity, but hopefully you get the idea. Augeas has turned the parsed lines in the sources.list configuration file into a logical tree structure that you can explore just like a filesystem. On my Ubuntu 19.10 system, it includes 225 lenses, so you can imagine that it will provide a fairly comprehensive view of your system configuration. There are lenses for CUPS, GRUB, Apache HTTP Server, MongoDB Server, and many more.
So how does Augeas fit into osquery? Osquery includes an augeas table that connects the Augeas API into the osquery SQL interface. Instead of running augtool, you can use osqueryi or scan remotely using the osqueryd distributed read feature.
Why does this matter?
First, you can now access configuration files from hundreds of different applications through osquery without creating a new table or writing a line of C++. You can even add support for your internal applications by writing a lens that parses your proprietary configuration file in the Augeas lens domain-specific-language (a cross between AWK and YACC), which should be much faster than implementing a full parser in C++.
Second, if you’re comfortable with SQL, you can do a lot more expressive searching of your configuration data through the osqueryi interface than you can with augtool. In my augtool example I piped the output through two rounds of grep to get something to include in this blog, while in my osquery example I just added a couple of extra ANDs to my WHERE clause. You can even join data here with other tables whenever that would make sense.
And of course with osqueryd connecting to osctrl or Uptycs, you can run these Augeas queries across your entire fleet of endpoints in a matter of seconds, confirming that your fleet is properly configured.
Here at Uptycs we’ve been working on a few improvements to osquery Augeas support.
Typically, Augeas loads the entire set of lenses at startup time in order to determine which lenses can be used to parse which files. Each lens specifies a set of include and exclude paths for files it believes it should be capable of parsing. At Uptycs we’ve found that loading all of the lenses may use more CPU and memory than desirable, and many of our customers may have their configuration files located in non-default locations.
In order to remedy this, we’ve added an optional “lens = ” clause to the Augeas table. This allows you to only load a single lens, as well as use a lens to parse a file in a non-default location. It’s a faster and more flexible application of Augeas.
We’ve also been thinking a lot about container workloads. Augeas is one of many tables we’ve adapted to quickly operate inside of containers and even join to other container-related tables to include information about the containers themselves.
So that’s it. Using Augeas with osquery allows you to scan the configuration of your entire fleet of hosts, easily adapting to new internal applications and their proprietary configuration files. And with Uptycs, you can even extend your queries from hosts into each and every running container across your infrastructure.
Schedule a demo and learn how the combination of Uptycs and Augeas can help your organization.
Image by TheLight from Pixabay.