In this blog post I’ll cover osquery’s ability to provide performant behavior and its capabilities to excel at enterprise grade requirements. Many observations covered in this blog will highlight various capabilities of osquery that should aid in your journey toward an enterprise-grade osquery deployment.
As discussed in another blog post, the osquery agent provides a rich and structured source of telemetry. The open-source project is well documented and there are numerous osquery tutorials to get you started. In this blog post I’ll introduce some key topics for consideration in enterprise-wide deployments:
- Osquery HTTP/TLS-based deployment
- Osquery vs other script-based data collection options
Osquery HTTP/TLS-based deployment
There are multiple ways to configure and deploy osquery in production. Some of the more popular approaches are summarized in the table below:
Option 1 is the simplest to implement, however, you are then reliant on your config management tool for control plane management and by default your config management tools will lack an understanding of controlling osquery. Additionally, you will need a separate solution for log forwarding and ingestion to a destination.
Option 2 has greater complexity than Option 1, as you need to deploy a fleet manager and ensure log forwarding is in place.
Option 3 is a more modern approach to osquery control and data plane management and can scale extremely well, as it is based on HTTP/TLS techniques for web scale deployments. This option can provide deterministic performance by place shifting complex detection analytics to a backend.
In this blog post, I’ll discuss the HTTP/TLS capabilities of osquery. Osquery has an interesting trait with respect to using HTTP/TLS connectivity. That is, it behaves like a web browser and connects outbound to a backend web service. In this blog I’ll outline alternate solutions that are prevalent today and then introduce the differences with osquery HTTP/TLS connectivity.
Osquery vs other script-based data collection options
Osquery HTTP/TLS behaves in the opposite manner of most script-based data scraping implementations. With script-based data scraping, the server connects to an endpoint/host via ssh or an agent connection to ask questions via a script, with the responses sent to a log file. Most Powershell and DSC scripts are implemented this way. That is, you have to connect to each machine and launch the script and then push the locally generated logs to a backend such as Splunk or ELK.
Unlike the above implementations, osquery can be configured to work in exactly the opposite way with the endpoint initiating an outbound connection over HTTP/TLS to the backend using REST semantics. The endpoint accepts query/work from a backend and posts the response back to the backend like any web browser. The same is the case for events and scheduled queries. With this implementation of osquery you get all the benefits of a mature web services environment with high-performance implementations of HTTP/TLS, highly secure transmission of data, and scalability based on modern distributed system architectures.
Osquery can be configured to be highly performant for multiple security use cases by using osquery as a source of telemetry and place-shifting your security detection pipeline and complex correlation to a scalable backend. The backend correlation may also include external data like third-party threat intelligence. While the data is in flight, it can be pushed through streaming analytics pipelines for high-speed correlation and alerting. Additionally, osquery has tooling to analyze the cost of queries and built-in mechanisms to stop running resource intensive queries.
The above characteristics lay the foundation for deploying, controlling and analyzing the data from osquery at scale in an enterprise.
Related osquery resources:
