The Restream Queue is at your disposal - but how do you best utilize it?
- Identify that a Problem Exists - You may realize that events are going to the restream queue in a number of ways.
- You may realize that events you expect to be in your target data warehouse don’t arrive - the Restream Queue should be the first place to check.
- You may notice an increment in the Restream count on the plumbing page or in the event list of the Restream Queue page.
- You may receive a digest email (configurable) which indicates events are erroring out.
Understand and Address the Problem(s)
- The Restream Queue page will show you the events that have errors along with the error reason. You can sort and filter by the input the event came from, the event type, the type of error, timeframe, or do a free text search to understand the events.
- Based on the error reasons, there are different ways to handle each type of problem. If you need help understanding an issue, reach out!
- Make the change in the Mapper, Code Engine, or your target data warehouse to address the source of the error. You can click to navigate directly to the relevant area in the Mapper or bring the event as a sample in the Code Engine from the right pane of an expanded event row.
Restream – Restream the events into your target database by clicking the Restream button in the Restream Queue screen.
- Any event that still generates an error is simply stored in the Restream Queue again.
- You can restream events as many times as you like without affecting the processing of newly arriving events. A metadata field
[metadata][restream_count]will indicate the number of times this event has been restreamed.
- We recommend resolving the Restream Queue iteratively, meaning that you should first handle one or a small set of problems that caused a significant set of events to be sent to the Restream Queue, then move on to the next problem(s).
Should your events loaded into your target data warehouse, but you decide you'd like to change the way that loading occurred, Alooma also has an S3 Retention feature that enables you to redo or change the loading of events that have already loaded into your target data warehouse.
Those are the basics of restreaming - read on about some common event problem reasons and solutions.