Summary: Common questions and answers regarding the use of the Restream Queue.
The Restream Queue page (accessible via the Restream Queue button at the top right of the Plumbing page, or via the left navigation pane) shows a breakdown of all the events in the Restream Queue.
Sortable and filterable by input, event type, error type, timeframe, or free text search.
Breakdowns of the contribution to the entire queue from different sources or errors, with a timeline chart.
The ability to see the error message and the event itself.
The ability to go directly to the relevant area in the Mapper or bring the event as a sample to the Code Engine.
First of all, don’t worry, your Restream Queue is monitored to ensure you are notified if it is starting to fill up beyond a certain threshold. This could be caused by importing a new source with a high volume of data and not mapping these fields to a table in your data destination, for example. Another potential reason events are sent to the Restream Queue is if a data warehouse table was deleted or Alooma cannot connect.
Although unlikely, if you still do not have a chance to address any of our emails, phone calls, IMs, or smoke signals and the Restream Queue does fill to capacity, any further events will be discarded. However, if S3 Retention is enabled, you can still recover your raw events and there will be no data loss.
We get it, not all events are created equal and not all must reach your data destination. You have a few options:
If there's a set of events which you want to clear out of the Restream Queue, add code to the Code Engine that identifies them by some attribute and then discards them. Then, restream. Note that newly arriving events with that attribute will also be discarded.
If you don't care about any of the events currently in the Restream Queue, because you've addressed the events in there that you do care about, you can click the Restream button and then choose the Purge Restream option. Note this is a destructive operation and will permanently delete all events from the Restream queue.