Record Streams

The concept of a record stream is a solution that solves the problem of out-of-order data ingestion of inter-related data. A stream enables Tendo’s Object Service to handle records as a collection rather than a single record so they can be processed in a single operation.

When bulk records are loaded into Tendo from a customer’s Electronic Health Record (EHR), the order in which they are loaded can be important because of relationships between records. To properly hook up those relationships, any record that is referenced by another record must be ingested in a specific order.

The Record Stream API offers a solution to the ordering problem by buffering the records until they have all been collected, enabling resolution of references between records of the same ingestion.

Stream types control how Object Service handles each record mutation that it receives for a stream. There are two types:

Buffered

A buffered stream will stage every record mutation that it receives for the stream. Upon finalizing the stream, it will attempt to resolve any inter-stream references and execute the record mutations. If the stream is transactional, it will treat all of the record mutations as a single atomic operation.

Auto Commit

When records are provided to objects, the system will check to see if the referenced values exist. If so, it will create the record. If not, it will put stage on the record. Upon finalizing the stream, it will attempt to resolve any inter-stream references and execute the record mutations.

To view Record Streams, navigate in Tools to Data and click on the Record Streams tab at the top of the page.

Under the Record Streams tab, you can see a table with the following information about each record stream - Its Reference, Status, Creation Time, Type (Buffered or Auto Commit), when it was Finalized and Completed, and its entities.

Click on the filter icon at the top right to filter the record streams by their Status (Active, Complete, Expired, Finalized, or Initialized), their type (Auto Commit or Buffered), the date and time from and to finalization, and whether they have entities.