Redshift set VARCHAR(256) to VARCHAR(MAX)

[eakmanrq]

If Redshift gets a VARCHAR column of an uncertain precision it will default the value to 256. For users who aren't explicitly defining columns to types or casting columns this can be annoying having to remember to cast to VARCHAR(MAX) in order to override the default.

This change would have SQLMesh detect when a VARCHAR(256) is being created and default to VARCHAR(MAX) instead. https://docs.aws.amazon.com/redshift/latest/dg/r_Character_types.html#r_Character_types-varchar-or-character-varying

In order to complete this task, the Schema Differ needs to be updated to be able to detect precision changes between VARCHAR columns and alter if the precision is increased.

[jmarch]

From my experience, this might be a dangerous path to take...

When I was using Redshift (years ago), there was a hard limit on the maximum row width in the query processing engine. For wide result sets with many VARCHAR columns, it was fairly easy to reach that limit. I also found a significant cost in IO/memory use when using unbounded VARCHAR lengths. Some queries would take "forever", but when typed more strictly would run just fine.

So before I'd presume you can MAX all VARCHARs by default, I'd suggest running some basic tests on a Redshift instance to see how many columns you can use before hitting these limits.

All this said, I tried to find the limit in their docs just now, but was unable to locate it. Perhaps they've improved the engine since I last had these issues.