As infrastructures get ever more complex, managing security becomes a significant issue. Alerts and logs are coming from many different systems, in as many different formats, and it’s important that the right information is delivered to the right person in order to make the right decision to prevent a security breach.

That ‘right time’ information model is critical, and it needs tooling that can bring all these information sources and events into one place. Security Information and Event Management, SIEM, is a rapidly growing part of the enterprise security market, building and delivering smart security dashboards that analyse and prioritise these messages, using a mix of log file analysis and machine learning. In a complex threat environment, modern data centres need a SIEM to operate effectively, sat next to your application and network monitoring tools and helping manage your response to incidents and warnings.

However, there’s a problem when it comes to cloud infrastructures: you may not have full visibility into all the elements of your environment, especially if you’re building on top of service and platform elements. Someone has the information that’s needed to secure your applications, but in many cases that isn’t you — it’s someone in the hyperscale cloud’s network operations centre.

Introducing Azure Sentinel

Microsoft recently launched Azure Sentinel, its approach to modern SIEM. Working across on-premises and in-cloud infrastructure, it’s intended to be easy to set up, low maintenance, and easy to use. By building on cloud-scale data collection, and on Microsoft’s own threat detection tools, Azure Sentinel can automate response using orchestration across your entire estate. It’s software-as-a-service so it’s scalable, and you only pay for the resources you use.

SEE: Vendor comparison: Microsoft Azure, Amazon AWS, and Google Cloud (Tech Pro Research)

Perhaps Azure Sentinel’s biggest advantage is its support for Microsoft’s security graph, as well as proven tooling inside Azure that’s part of Microsoft’s own security analytics platform. The security graph alone currently processes millions of signals a day, working across all of Microsoft’s cloud-hosted platforms to develop models of how attacks progress — even when they may be slow advanced persistent threats, where actions are normally hidden in the noise of a busy data centre’s operations.

One of the key elements of Azure Sentinel is Azure Monitor, a component of Azure’s application monitoring platform. Able to ingest petabytes of log data every day, it’s part of Azure’s DevOps framework; adding security data moves it into the SecOps space, and using tools like Azure Data Explorer and its Kusto query language makes it easier to build and construct your own queries.

Getting started with Azure Sentinel

Getting started can be relatively quick. First add a Log Analytics workspace to your Azure account. Once that’s up and running, you can enable Azure Sentinel from the Azure Portal. You’ll need to add a workspace, which is where all the data associated with your subscription will be stored. You can have multiple workspaces in an account, but each workspace is isolated.

Once the service is up and running, your next task is configuring connections to services, apps, and machines. One thing to note: this is not an agentless tool — you must install the Azure Sentinel agent on all physical and virtual machines you’re monitoring. Apps will need to provide logs that can be shipped via the familiar Linux Syslog server, running on a VM with an agent that forwards logs to your Azure Sentinel workspace.

For Azure services, the process is a lot easier. From Data Collection, choose the service you want to monitor. Logs will then start streaming into Azure Sentinel, ready for analysis. Usefully the connection process provides lists of recommended dashboards, so you can quickly set up an analytical view of your infrastructure.

Viewing and analysing security data

Once data is flowing into Azure Sentinel, it will quickly start populating its dashboards. These mix Azure’s existing security and analytical tools with tables and charts. Any incidents are grouped into cases, bringing together related alerts into a single view. By taking this approach, Azure Sentinel aims to reduce the feeling of information overload that you get when presented with seemingly unrelated alerts. Events are charted across a 24-hour period, allowing you to compare today with yesterday. You can drill down into cases to start an investigation, with a map view showing the source of malicious events and where exfiltrated data is being extracted.

Built-in dashboards support Azure activities, Azure Active Directory, and your on-premises servers. Other information displayed comes from applications, from Office 365, and from third-party hardware, including firewalls and other security appliances and services. For example, Azure Sentinel offers two different Azure AD dashboards, one examining sign-ins and the other exploring its audit logs. Both offer important insights: one shows possible attacks and one indicates accounts that are unexpectedly moving groups or gaining privileges.

Watching data flows in and out of your network can quickly show if there’s been a breach, and Azure Sentinel can pinpoint associated events and alerts, with machine learning-powered systems notifying you of operational anomalies that need investigation. Anomaly detection is an important function, and it helps identify new attack vectors or long, slow data exfiltrations, using models developed at Microsoft. If an anomaly is detected, then you should investigate activities across your network around that time period.

Customising security

Azure Sentinel has a dashboard creation tool where you can add your own new visualisations, building queries and using them as the source for graphs and charts. Queries are written in Microsoft’s Kusto query language, so you can use tools like Azure Data Explorer to build and test new queries. As Kusto is designed to build queries that can work across a mix of structured and unstructured data, it’s an ideal tool for working across many different log formats, bringing them together in a single view.

SEE: Windows 10 security: A guide for business leaders (Tech Pro Research)

Additional detections can be found on the Azure Sentinel GitHub community, and can be added to new data sources. Detections are developed inside Microsoft, and form the basis of rules used by Azure Sentinel to generate alerts. You can use these as the basis for your own, fine-tuning them to support new sources and new log file formats.

Using Azure Sentinel to hunt threats

With Azure Sentinel monitoring your systems, you’re ready to start using it to hunt threats. With a considerable amount of data to analyse, the default queries and dashboards can’t find everything. Instead you can use ‘hunting queries’ to make your own exploration, with sample queries to get you started and a query language you can use to modify existing queries and create your own. Interesting data can be bookmarked, helping you build cases. Hunts can be recorded step by step in notebooks, taking a lesson from the analytics world. Azure Sentinel notebooks can be passed on to other investigators, playing the queries back so they can see what you’ve found and how you found it. Microsoft’s Incident Response team will provide additional notebooks via GitHub.

The preview of Azure Sentinel is currently free (and has no SLA), although you will need to pay for some features — Azure Monitor, any machine learning customisation, and workflows using Logic Apps, for example.

Microsoft has quietly been building out its security product line, and Azure Sentinel is the latest in a line of tools that work form consumer to cloud. By building on the skills and lessons of its own security teams, the result is a set of tooling that’s appropriate for day-to-day security monitoring and for active threat hunting, across hybrid on-premise and cloud environments. It’ll be interesting to revisit Azure Sentinel when we know the final pricing model. Until then, it’s well worth a look, and running the trial could teach you some important lessons about your network.

Subscribe to the Microsoft Weekly Newsletter

Be your company's Microsoft insider by reading these Windows and Office tips, tricks, and cheat sheets. Delivered Mondays and Wednesdays

Subscribe to the Microsoft Weekly Newsletter

Be your company's Microsoft insider by reading these Windows and Office tips, tricks, and cheat sheets. Delivered Mondays and Wednesdays