Today’s enterprise applications are increasingly deployed in various public clouds. Even if your organization is traditionally based on on-premises infrastructure, chances are that you will have to consider migrating to a public or hybrid cloud model. With this transition to public cloud, there is often a lack of clarity about a key question related to security: who is responsible for your application’s security and compliance with established standards. In the case of public clouds, the answer is that security is a shared responsibility between the public cloud provider as well as your organization that deploys in the public cloud. Let’s double click on that shared responsibility to identify some aspects of security that can become blind spots and cause vulnerabilities.

Typically, a cloud provider will be responsible for ensuring that the infrastructure is secure. This will include providing identity management, authorization controls, policing network accesses from outside, tracking changes made to infrastructure, and ensuring timely infrastructure software patching to mitigate known vulnerabilities. Application owners on the other side will have to ensure that they use these public cloud provider’s security knobs appropriately to safeguard their deployments as well as customer’s data. In addition, application owners will have to follow the security best practices for their own development as well as any third party or open source code that is part of their cloud deployment. While there are multiple security aspects that need to be addressed by enterprise application owners, let’s focus specifically on security monitoring.

Security monitoring plays an active role in not only detecting threats but also identifying any violations of an organization’s internal policies. For instance, an organization that is responsible for storing and processing their customers' sensitive financial data must keep an audit trail of who accessed them and at what times. If an employee in the organization accesses those data at a time when their role did not require such an access, it is a potential breach of the policy and can undermine the organization's overall security posture. Various security compliance standards require retention of such audit trails for specific lengths of time.

An enterprise with public cloud deployment must answer two fundamental questions about their security monitoring:

• Are we collecting the right information from cloud provider’s infrastructure as well as their own service deployment to uncover any security threats or policy breaches?
• What are we doing with these collected observability data? Are we simply storing these data to fulfill compliance requirements or using them to actively detect and respond to threats?

If an organization is either not collecting appropriate observability data or not using it actively, it will be exposing itself to security “blind spots”. It is a known fact that most security holes are created due to policy violations by organization's own employees without any malicious intent. It is therefore critical to identify such breaches early, so that they can be corrected before attackers could discover them as security holes. Let’s look at both collections as well as active detection/response challenges with some technical details.

Security Observability

Enterprise applications are increasingly using Kubernetes (K8S) with containerized micro services deployments in public clouds. Some applications prefer virtual machine instances based deployments as well. In both cases, application owners can deploy collection agents to capture required audit and access logs. These logs are forwarded to their own or a third-party provider’s monitoring and analytics platform to fulfill their security needs.

In addition to deploying to VMs and K8S, a typical enterprise application can utilize a number of native resources from the public cloud. These native resources include serverless compute, block storage, databases, identity and access management, audit logs, threat logs, and a number of data warehousing and analytics services. Unlike VMs or K8S clusters, application owners cannot directly deploy their observability data collection and forwarding agents on these native resources. Cloud providers typically collect all types of observability data from their native resources and forward them to their own native monitoring system.

While above is a typical deployment scenario, it results in fragmented observability data in an organization’s own monitoring setup as well as in a cloud provider’s native monitoring platform. This fragmentation not only makes the job of a security operations team harder but also fails to provide a “single pane of glass” for holistic threat detection and response. This is what we have been referring to as “blind spots” in security observability. In the following section, we elaborate on our solution to overcome it.

Data Integration for Detection and Protection

Instead of keeping data and running analytics in two disjoint monitoring platforms, it is possible to integrate relevant security observability data for a holistic security incident detection and taking prompt action to mitigate it. This objective can be achieved by connecting cloud provider’s native monitoring system with the customer's own monitoring platform.

Connector service can allow an organization to identify their required logs, metrics, or events from a cloud provider’s native monitoring system that need to be exported out to its own analytics backend. Connector will ingest, schematize, and forward these observability data on behalf of that organization. For an organization’s analytics backend, these data will have the same format as their other security monitoring data to allow a consistent analysis, detection, protection, and compliance framework.

How can we help?

Cloudvista can help in both collecting and integrating all security related observability data. We can integrate with customer’s existing backend or build an entire backend platform that will be extensible as they grow. An end-to-end observability architecture that includes all possible data sources, a consistent internal data format, and an extensible analytics engine will ensure an enhanced ROI as well as a standards-compliant security posture.

For more information on how Cloudvista can fulfill an organization’s security data collection requirements while removing any blind spots, contact us at email : info@cloudvista.net