Sign in

Introduction to Notification Service

notification service

Purpose:

  • Solve synchronisation and communication needs between our Privacy Services and your systems by notifying clients of the platform when any data changes (privacy requests, permission changes, etc)

Main goals:

  • Provide notifications of changes/actions through APIs, event streams or web hooks to clients, so they can act on these changes and synchronise their systems with DPS
  • Store and maintain status of privacy requests from end user to services

    • Consent changes (approved/revoked)
    • Objections to data processing purposes
    • Requests for data access/portability and erasure

Notification Service design overview

Southbound requests

Southbound requests are privacy requests that are triggered by the end user, via DPS, to the clients that are affected by the request.

These requests must be relayed to the clients that are affected by the request and need to take action on them. Southbound requests may generate equivalent responses from clients, affecting their status.

Northbound requests

Northbound requests can be of two types:

  • Privacy requests initiated by clients to DPS (on behalf of a user)

    • Mainly objection and consent requests that are triggered directly by a client
  • Response to privacy requests previously initiated by the end user via DPS

    • Mainly responses and updates to the originally triggered southbound request from DPS to clients

Any privacy requests triggered or updated in DPS may trigger a notification to the parties affected so they can keep all their systems in sync. The main goal of the notification is not to provide state changes, but to notify clients that changes have happened and allow them to act on these changes.

This can happens in two ways:

  1. Through a notification from DPS to the systems affected

    • Webhook (HTTPS POST call)
  2. Through API calls to DPS from the systems themselves

    • Privacy requests APIs

Notification design patterns

There are three basic patterns that can be used for notifications:

  • Polling

    • Simple to use
    • Can lead to complex scheduling
    • Trades efficient use of resources for fresh data
  • Ping/Pull

    • Avoids scheduled pulling
    • Consumer must host an endpoint
    • Moderate latency (3 hops)
    • Complex, although common (Facebook RealTime updates for example)
  • Push

    • Content of change is pushed
    • Payload can be light, diff or full resource
    • Consumer must host an endpoint
    • Lowest latency (1 hop)

Our design decision for notifications have been to do Ping/Pull in this first iteration.

In future iterations we’ll might explore Push in 2 waves (Light payload then Full payload), based on the http://resthooks.org/ defined patterns.

In addition, clients themselves will also be able to do Polling via the APIs provided if Ping/Pull is not desired.

High level notification framework (draft)

notification flow

When a change or privacy request is triggered via the privacy API, the event is published on a topic per project for all the affected projects of that privacy request.

  1. The API stores the request for that client & user combination
  2. It then publishes a notification of that privacy request to one or many topics (per project + privacy request types)

    • The notification then triggers a lambda function
  3. The lambda function starts processing the notification

    • Identify the notification endpoint and authentication method
    • Post the message to the specified recipient endpoint
  4. The lambda function then update status of the request based on the response from endpoint

    • Acknowledged or failed
  5. The client does an API call to retrieve the payload of the privacy request
  6. The API retrieves the privacy request
  7. The API returns the privacy request and its latest state + payload

The privacy request database will have the status from any initiated request processed or not processed by the lambda function. Failed and unacknowledged requests can then be processed separately (retry/abandon functionality). The same database is used to process southbound API requests from clients, for example when clients are updating the privacy requests status with completed.