Performance¶

https://spark.apache.org/docs/latest/sql-performance-tuning.html

https://sparkbyexamples.com/spark/spark-performance-tuning/

https://engineering.salesforce.com/how-to-optimize-your-apache-spark-application-with-partitions-257f2c1bb414/

• caching

• sql hint

• partition

• dynamic repartition

• dynamic resource allocation