VED-982: Create POST MNS Notification

[Akol125]

Summary

• Routine Change

• Summary
  Implemented MNS integration to publish real-time CloudEvents notifications for immunisation record changes (create/update/delete).

Key Components

• Lambda Handler
  Processes SQS events from DynamoDB streams
  Publishes to MNS with partial batch failure handling
  Structured logging with trace IDs (message, immunisation, notification)

• Notification Creation
  Builds CloudEvents-compliant payloads from DynamoDB stream data
  Fetches GP ODS codes from PDS
  Calculates patient age at vaccination
  Populates filtering attributes (organisation, application, vaccine type, action)

• Utilities
  Recursive extraction from nested DynamoDB events with type descriptor unwrapping
  MNS service class for API interactions (subscribe, publish, delete)
  Enum-based field mappings for type safety

• Testing
  Unit test coverage using real SQS payloads
  Mocked external dependencies (PDS, MNS)
  Comprehensive edge case and error handling tests

Add any other relevant notes or explanations here. Remove this line if you have nothing to add.

Reviews Required

• Dev
• Test
• Tech Author
• Product Owner

Review Checklist

ℹ️ This section is to be filled in by the reviewer.

• I have reviewed the changes in this PR and they fill all of the acceptance criteria of the ticket.
• If there were infrastructure, operational, or build changes, I have made sure there is sufficient evidence that the changes will work.
• If there were changes that are outside of the regular release processes e.g. account infrastructure to setup, manual setup for external API integrations, secrets to set, then I have checked that the developer has flagged this to the Tech Lead as release steps.
• I have checked that no Personal Identifiable Data (PID) is logged as part of the changes.

[github-actions]

This branch is working on a ticket in the NHS England VED JIRA Project. Here's a handy link to the ticket:

VED-982

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
96.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[dlzhry2nhs]

If we are using batchItemFailures (I am in favour of us doing this as it is a good pattern), then we need to switch this on in the terraform. You can do this in the event source mapping for the Lambda function.

You will need to do this in order for this to work. It might be good to take a look at some test results. E.g. if the secrets are not configured for now, then we would expect item failures to be raised and results to be posted to the DLQ. This would be a good test.

[dlzhry2nhs]

There is no such thing as dev-blue. It may be more informative to mention int-blue - perhaps there is already a description for this variable at the higher level?

[dlzhry2nhs]

Some suggestions. Happy to pick up on Monday if anything does not make sense.

Some pretty major findings that mean it fundamentally won't work. And I think there is a lot of opportunity to make this new Lambda a bit cleaner. But a good start nonetheless.

[dlzhry2nhs]

At some point we should tidy up IdSync as there are quite a lot of bad patterns in here. However, I would prefer if we keep the IdSyncException in here - it is specifically for some strange handling that has been built in here that I would like us not to bleed into our new feature.

Or at the very least, we return it. If you fancy we could also improve the IdSync if you want to as part of this work.

[dlzhry2nhs]

Please remove or uncomment, depending on what you were planning to do with this.

[dlzhry2nhs]

Please leave this in ID Sync. This is a weird wrapper used over there.

For our new service it would suffice to import the pds_service and use the relevant method for it.

[dlzhry2nhs]

I am not keen on the new Lambda using this custom exception. It has nothing to do with Sync. The PDS sync process is the Id Sync SQS + Lambda process.

The new Lambda is just running a PDS get to retrieve data and enrich the outgoing event.

[dlzhry2nhs]

MNS_BASE_URL?

[dlzhry2nhs]

As a minimum, I expect us to only have one line within the try catch. A function called something like process_record would suffice.

It would be even better if we could use a class-based approach and try to abstract away all the complexity of SQS -> Lambda batch failure responses. Happy to discuss online, but if we go down such a route, it could be really nice to have a base class that maybe handles obtaining the generic fields from the event, exposes a process_records method, and also handles the batchItemFailures response structure.

It could have an unimplemented process_record method that inheriting classes implement. In our case, that would be the MNS Publishing business logic. This will make things much nicer.

You could then bind things to the class that we need such as relevant clients (MNS and PDS). I would argue there should be a standard MnsPublisherService which the Lambda Handler class could use that would work purely in terms of business objects i.e. not care about the external SQS event structure. But that might be slightly too much work. As a minimum, we need the first approach, but really worth considering restructuring with a bit more thought.

[dlzhry2nhs]

Rather than having constants for the dict keys, could it be worth having TypedDicts or something more stringent?

If not for the imms data, certainly for the outbound MNS payload.

[dlzhry2nhs]

Log is incorrect.

[dlzhry2nhs]

The returned value will be an array, if it exists. This code will blow up.
Please do not guess. Ask if you are unsure, or read the API documents: https://digital.nhs.uk/developer/api-catalogue/personal-demographics-service-fhir#get-/Patient/-id-

Having looked at the docs, I think we also need to check the end key to ensure we only retrieve the GP ODS code if the registration is still active. We do not want old practices to get notifications if a registration has been ended.

[dlzhry2nhs]

This will not work. You have to have an instance of a class. Because of the way authentication has been built in currently, you have to instantiate each time with an authenticator.

I would advocate looking at Matt's changes for the performance test. Either that will merge first and you can start using it, or you could replicate it here. But either way, you need to fix the fundamental issue raised above.