Error handling - insert error information into a table

Help Connectors and Pipelines
1

Hello,

We have a pipeline which inserts event data from kafka into several tables via execution of several stored procedures. We need to store failed batches errors into a table of errors. How can we extract the error from a failed procedure?

We want to do something like this:
INSERT INTO [LearningErrorLog]
(
ErrorNumber
,ErrorDescription
,ErrorProcedure
,ErrorState
,ErrorSeverity
,ErrorLine
,ErrorTime
)
VALUES
(
ERROR_NUMBER()
,ERROR_MESSAGE()
,ERROR_PROCEDURE()
,ERROR_STATE()
,ERROR_SEVERITY()
,ERROR_LINE()
,GETDATE()
);

I see the singlestore studio auto completes the function ERROR_MESSAGE, but then says it’s not defined.

Any help would be appreciated.

2

Hi Maximg and welcome aboard! :wave:
We’d love to help with this issue.
Are you running on the managed or self-hosted service and what version number?

3

Hi Maria,

I’m running in the self hosted currently.
SingleStore DB Version: 7.8.14
Studio Version: 4.0.7

Thanks

4

hi @maximg did you check the pipeline error table that already captures the errors messages ?

Regards,
Manish Kumar

5

Hello @mkumar, no the insertion errors are not present there. what about my original question, getting last error information?

6

Hi Maximg. Hope you find this helpful!

You may be able to get the information you need within the information_schema and insert those values into another table as needed.

All errors from pipelines can be viewed on a table called

information_schema.PIPELINE_ERRORS

Here is a useful query to show all of the errors that occur in the pipeline:

SELECT DATABASE_NAME, PIPELINE_NAME, ERROR_UNIX_TIMESTAMP, BATCH_ID, PARTITION, BATCH_SOURCE_PARTITION_ID, ERROR_KIND, ERROR_CODE, ERROR_MESSAGE, LOAD_DATA_LINE_NUMBER, LOAD_DATA_LINE FROM information_schema.PIPELINES_ERRORS;

Here is page that goes over how to handle pipeline errors.

If you can share what worked for you, others in the community may benefit from your postings. Cheers :beers:

7

Hey @mkumar and @MariaSilverhardt,

Thanks for the support. The default pipeline errors are not enough. I want to add custom data that’s available during the error handling.

That’s why I’m asking about a way to extract the error data. To add it to the custom stuff.

Cheers, Maxim

8

Hi Maxim, can you please paste your code and the ‘not defined’ error that you are getting back here? this will help us determine the issue you are having

I see the singlestore studio auto completes the function ERROR_MESSAGE, but then says it’s not defined.

1 Like