The Evidence Collection Challenge
During audits, security teams spend countless hours gathering evidence—screenshots of configurations, exports of policies, logs demonstrating control effectiveness. This manual process is time-consuming, error-prone, and produces evidence that's outdated almost immediately.
FedRAMP 20x Removes the Choice
Automated evidence collection used to be a maturity goal. Under FedRAMP 20x it is a structural requirement, because the program changed what counts as evidence in the first place.
- Narrative is no longer sufficient. 20x moves the burden of proof from prose describing a control to machine-readable artifacts demonstrating it. A paragraph asserting that encryption is enforced does not satisfy what a screenshot of the actual key policy does.
- Continuous replaces point-in-time. An annual evidence sweep cannot feed a model that expects current validation state. If your collection is a once-a-year project, you have nothing to report between cycles.
- Machine-readable output is the interface. Evidence has to arrive in a structure that maps to control statements automatically. Hand-assembled folders of PDFs do not.
- Volume makes manual collection impossible. Continuous validation across a full control baseline generates far more artifacts than a team can gather by hand, regardless of headcount.
The practical consequence: you cannot meet 20x by working harder at the old process. The pipeline is not an efficiency play, it is the only shape of the work that fits.
Where GRC Platforms Stop
Most organizations start with a GRC platform—Drata, Vanta, or similar—and they are genuinely good at what they cover. Policy lifecycle, SaaS configuration checks, security awareness tracking, and evidence for the common commercial stack are all solved problems in those tools.
The gap shows up at the edge of the connector catalog. These platforms were built for SOC 2 on commercial cloud, and a FedRAMP boundary rarely looks like that:
- GovCloud, air-gapped, and on-premise components with no supported connector
- Bespoke internal applications that hold in-scope data but expose no standard API
- Control families like contingency planning and physical security where the evidence lives in another team's system
- Inherited controls from a hosting provider that need to be represented, not tested
The honest position is that a GRC platform gets you a meaningful share of the way and then stops. What matters is what you build to cover the rest.
Three Ways to Cover the Gap
There is no single right answer here—the choice depends on what your team already runs and who will maintain it in two years. These are the three approaches we build and the trade-offs we would tell you about before starting.
1. A custom evidence service
A small internal web application that sits between your bespoke systems and your GRC platform, exposing evidence in the shape the platform expects.
- Best when: you have in-scope systems no vendor will ever build a connector for, and you need full control over what gets collected and how it is attested.
- Trade-off: it is software, and software needs an owner. This is the option that fails quietly if nobody is accountable for it after the initial build.
2. A dataflow or workflow engine
Apache NiFi and Tines solve overlapping problems from different directions, and the right pick depends on the evidence.
- NiFi suits high-volume artifact movement—log samples, configuration exports, scan output. Its built-in provenance tracking is genuinely useful for compliance, because you get a defensible chain of custody for every artifact as a property of the platform rather than something you have to build.
- Tines suits API-driven collection with decision points and human approval steps. If evidence gathering needs a person to confirm something before it is recorded, this is a better fit than a pure dataflow tool.
- Trade-off: another platform to run and license. If you already operate one of these for security operations, the marginal cost is low. If not, you are adopting infrastructure to solve a documentation problem.
3. CI/CD pipelines you already run
GitLab CI, GitHub Actions, or equivalent, running collectors on a schedule and committing artifacts to a repository.
- Best when: your infrastructure is already defined as code and your team lives in the pipeline. Evidence collection becomes another scheduled job rather than a new system.
- Underrated benefit: the repository gives you an immutable, timestamped, signed audit trail for free. Git history is a better evidence ledger than most purpose-built tools, and assessors understand it immediately.
- Trade-off: pipelines are built for build and deploy, not long-term artifact retention. You will need a deliberate storage and retention design rather than letting artifacts accumulate.
In practice most FedRAMP boundaries end up with a combination: the GRC platform for what it covers well, pipelines for anything already expressed as code, and a purpose-built service for the systems nothing else reaches.
Building an Automated Pipeline
Our evidence collection pipeline automates the entire process from source systems to audit-ready artifacts:
- Source Integration: Connect to cloud platforms, security tools, and identity providers via API
- Scheduled Collection: Evidence is gathered on configurable schedules—hourly, daily, or on-demand
- Control Mapping: Each piece of evidence is tagged with the OSCAL controls it satisfies
- Artifact Storage: Evidence is stored with cryptographic hashes for integrity verification
Types of Evidence Collected
The pipeline handles various evidence types:
- Configuration Exports: IAM policies, security groups, encryption settings
- Log Samples: Authentication events, access logs, audit trails
- Screenshots: Automated capture of UI-based configurations when APIs aren't available
- Scan Results: Vulnerability assessments, compliance scans, penetration test outputs
OSCAL Integration
Evidence is automatically linked to OSCAL control implementations:
- Each artifact references specific control requirements
- Evidence bundles can be exported in OSCAL Assessment Results format
- Auditors can trace from control statement to supporting evidence
Audit Preparation
When assessment time arrives, evidence is ready:
- Pre-packaged bundles organized by control family
- Evidence freshness indicators showing collection timestamps
- Gap identification highlighting controls missing evidence
Results
Organizations using automated evidence collection report:
- 90% reduction in manual evidence gathering time
- Evidence always current—no more scrambling before audits
- Complete audit trails for every control
What We Build Around the Gaps
A healthcare technology company had Drata deployed and getting real value from it. Their SOC 2 evidence was largely handled. The problem was that their FedRAMP boundary included a GovCloud environment and two internal services that Drata had no connectors for, and roughly sixty controls depended on evidence from exactly those systems.
The proposal they had received elsewhere was to replace the GRC platform. We disagreed. Drata was doing its job well for the majority of controls, and replacing it would have thrown away working automation to solve a problem at the edges.
Instead we built collectors for the systems it could not reach and fed them into Drata through its API, so evidence arrived in the platform their team already used. Nobody had to learn a second tool or check two dashboards.
The GovCloud collectors ran as scheduled GitLab CI jobs, since their infrastructure was already Terraform and the team lived in those pipelines. The two internal services needed a small custom service, because neither exposed anything resembling an API and we had to build one.
Around eight weeks of work, two engineers. We came back for three weeks the following year when they added a second boundary, then stopped again. That is the working relationship we want—present when there is something to build, absent when there is not.
