Dry Run/Test Connection

I was just building a pipeline in DPC and using Create Table and S3 Load. Pretty fundamental components. I was working with a dataset I hadn't used before and I found myself going back and forth creating the table and getting the metadata correct. In the end I zipped back to METL and used the load generator tool. This is a really powerful and effective improvement for the user. It removes  a lot of friction by doing the heavy lifting for you. In general I feel the DPC needs more test/dry run functionality, the improvement on speed to value is superb. Is Dry runs/Testing  something we have on radar? Testing DB connections, Metadata definition, credentials etc.

We could build a “validator” that checked the table metadata before trying to load the data. This way customers can be confident they get up front feedback instead of waiting for the failure and re-running.

Thanks!