I have created S3 pipelines that load into a stored procedure. If I use the default statement to create the pipeline as documented, I get the error message: “Pipeline into stored procedure using modes other than IGNORE PARSER ERRORS and SKIP PARSER ERRORS’ is not supported by MemSQL”. Adding either the ignore or skip does work but we would prefer the pipeline to stop loading and shut off if it encounters a parsing error. We still have S3 pipelines that load directly into tables that work this way and would like to have the same behavior whether we load the data from S3 directly into a table or into a stored procedure. Is there any way we can get the same behavior for both types of pipeline?
1 Like
Hey!
What you’re asking for should be the default behavior. Can you show your create pipeline statement?
The version of the create pipeline that we get this error is this:
create pipeline s3p_dim_student
AS LOAD DATA S3 ‘[S3PATH]/’
CONFIG ‘{“region”: “us-east-1”}’
replace INTO PROCEDURE load_dim_student
FIELDS TERMINATED BY ‘,’
OPTIONALLY ENCLOSED BY ‘"’
ESCAPED BY ‘\’
I realized afterwards that the “replace” is left over from when we were loading this pipeline directly into a row store table and probably has no meaning here since we are dealing with duplicates in the stored procedure. Removing the replace gets rid of the error message and allows us to create the pipeline without the SKIP or IGNORE. However it does still seem that the error message is incorrect since I’m not sure what the replace would do with a stored procedure load anyway. Thanks for your help.
-David
Yep - it’s a bug with the text of the error message. REPLACE into a stored procedure is unsupported, and is in fact what triggered the message. We’ll fix the message shortly after 7.0.