Exception: Python in worker has different version 3.6 than that in driver 3.8

에러 내용: Exception: Python in worker has different version 3.6 than that in driver 3.8, PySpark cannot run with different minor versions. Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.

 

아래 링크를 참고하여 slave master 간 python 버전을 맞춰주었다. (worker가 3.6이였다) 

...그래도 안되서 찾다보니, 재작년에 같은 에러를 냈다. 그래도 해결안되서 분노의 재시동을 했더니 됐다 ; 

Install Python 3.8 on CentOS 7 / CentOS 8 | ComputingForGeeks

 

pyspark 에러