How to handle the single huge partition

[zuston]

Problem

In internal environment, some Spark jobs may exist data skew and it will make the local disk full in shuffle-server. How to solve this problem?

Best practise

1. Use the storage type of MEMORY_LOCALFILE_HDFS
2. Enable the storage manager fallback strategy. rss.server.multistorage.fallback.strategy.class org.apache.uniffle.server.storage.LocalStorageManagerFallbackStrategy

[zuston]

@xianjingfeng @jerqi Please check the answer of this problem. Thanks