Pipeline error handling

Help Connectors and Pipelines
1

It’s impossible to create pipeline with ERRORS HANDLE:
CREATE OR REPLACE PIPELINE pp_name AS LOAD DATA
LINK MY_S3_LINK ‘bucket/dir/*.txt’
SKIP ALL ERRORS
INTO TABLE tbl(col1, col2)
FIELDS TERMINATED BY ‘\t’
LINES TERMINATED BY ‘\n’
IGNORE 1 LINES
SET source = pipeline_source_file()
ERRORS HANDLE ‘data_import’;

Error Code: 1064. You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘ERRORS HANDLE ‘data_import’’ at line 1

Without last row I can create this pipeline.

Table has columns col1, col2, source - all text.

Exists any workaround to handle errors without such pipeline failure ?

Thanks!

2

ERRORS HANDLE is not supported in our syntax. What are you trying to do?

See CREATE PIPELINE · SingleStore Documentation
and look for the word “ERROR” – there’s a lot of content about various ways to skip or handle errors. Also, you could use pipelines to stored procedures to conditionally look for bad data and have your own procedural logic handle the errors.

3

Hi @hanson , thank you,
I look for possibility to create stable pipeline with diagnostocs on csv / tsv.
My users can sometimes create such files in Excel and send to S3, so, most stable way is parsing in procedure ((