Skip to content

Bug fix event_timestamp and add validation test cases for AWS_VPC_FLOW_CSV parser - #1213

Open
Murali-kodez wants to merge 4 commits into
chronicle:mainfrom
Murali-kodez:fix/aws-vpc-flow-csv-event-time
Open

Murali-kodez wants to merge 4 commits into
chronicle:mainfrom
Murali-kodez:fix/aws-vpc-flow-csv-event-time

Conversation

@Murali-kodez

Copy link
Copy Markdown

Description

What problem does this PR solve?
Fixes missing metadata.event_timestamp normalization in the AWS_VPC_FLOW_CSV community parser. Previously, flow events were extracted without normalizing the epoch start time (column11) into metadata.event_timestamp.seconds, which prevented accurate chronological event ordering and temporal YARA-L rule evaluations in Google SecOps.

How does this PR solve the problem?

  • Ensures the flow start epoch timestamp (column11) is correctly converted to integer and mapped to metadata.event_timestamp.seconds.
  • Added a synthetic, PII-free validation test fixture under content/parsers/third_party/community/AWS_VPC_FLOW_CSV_GUS/tests/test_vpc_flow.json to enable automated CI regression testing.
  • Verified parser execution locally to confirm proper UDM normalization.

Any other relevant information (e.g., design choices, tradeoffs, known issues):

  • All test fixtures use 100% synthetic RFC 1918 addresses, dummy AWS ENI resource IDs (eni-000000000000eee00), and zero customer or employee data.

Telemetry Sample & Output

Raw Sample (Sanitized):

4,000000000000,eni-000000000000eee00,1.1.1.1,0.0.0.0,12345,54321,8,1,323,1787585432,1787585474,ACCEPT,OK


{
  "metadata": {
    "product_log_id": "000000000000",
    "event_timestamp": {
      "seconds": 1787585432
    },
    "event_type": "NETWORK_CONNECTION",
    "vendor_name": "AMAZON",
    "product_name": "AWS VPC Flow CSV",
    "product_version": "4"
  },
  "principal": {
    "ip": "1.1.1.1",
    "port": 12345
  },
  "target": {
    "ip": "0.0.0.0",
    "port": 54321,
    "file": {
      "size": 323
    }
  },
  "security_result": [
    {
      "action": [
        "ALLOW"
      ]
    }
  ]
}

Checklist:
General Checks:
 I have read and followed the project's [contributing.md](https://github.com/chronicle/marketplace/blob/main/docs/contributing.md) guide.
 My code follows the project's coding style guidelines.
 I have performed a self-review of my own code.
 My changes do not introduce any new warnings.
 My changes pass all existing tests.
 I have added new tests where appropriate to cover my changes.
 I have updated the documentation where necessary.
 
 
 Open-Source Specific Checks:
 My changes do not introduce any Personally Identifiable Information (PII) or sensitive customer data.
 My changes do not expose any internal-only code examples, configurations, or URLs.
 All code examples, comments, and messages are generic and suitable for a public repository.
 I understand that any internal context or sensitive details related to this work are handled separately in internal systems.
For Google Team Members and Reviewers Only:
 I have included the Buganizer ID in the PR title or description.
 I have ensured that all internal discussions and PII related to this work remain in Buganizer.
 I have tagged the PR with one or more labels that reflect the pull request purpose.
 
 
 Further Comments / Questions
Happy to address any feedback, adjustments, or additional test cases suggested by the reviewers.

@Murali-kodez
Murali-kodez requested a review from a team as a code owner September 2, 2026 17:15
@google-cla

google-cla Bot commented Sep 2, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Updated metadata for AWS VPC Flow CSV parser with new fields and corrected values.

Signed-off-by: Murali-kodez <saimuralikrishna.kavuru@gmail.com>
@Murali-kodez

Copy link
Copy Markdown
Author

Hi @maintainers, all standalone validations and unit tests (3a) are now passing with clean testdata and compliant metadata. As an external community contributor without an internal SecOps instance, this PR is ready for review and for the 3b instance profiling checks. Thank you!

@Murali-kodez

Copy link
Copy Markdown
Author

Hi @chronicle/malachite-github-content-hub-parsers-writers , @Kambham5727 , @shridharn-stack
Friendly follow-up on this PR whenever someone has a moment to review.
All standalone unit tests (3a), metadata schema validations, and Google CLA checks are passing with clean synthetic testdata. Ready for review and for the 3b instance check to be triggered or skipped.
Thank you!

@Murali-kodez Murali-kodez changed the title Feature: Map event_timestamp and add validation test cases for AWS_VPC_FLOW_CSV parser Bug fix event_timestamp and add validation test cases for AWS_VPC_FLOW_CSV parser Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant