-
21/04/06 15:20:45 ERROR shuffle.RetryingBlockFetcher: Exception while beginning fetch of 1 outstanding blocks
2021. 4. 6. 15:22
pyspark 실행할 대 --master 설정을 해줘야한다... 하...
/ __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /__ / .__/\_,_/_/ /_/\_\ version 3.1.1 /_/ Using Python version 3.8.1 (v3.8.1:1b293b6006, Dec 18 2019 14:08:53) Spark context Web UI available at http://jiwon.local:4042 Spark context available as 'sc' (master = spark://jiwon.local:7077, app id = app-20210406152023-0003). SparkSession available as 'spark'. >>> numbers = sc.range(0, 1000000, 1, 2) >>> evens = numbers.filter(lambda x: x % 2) >>> listfelements = evens.collect() 21/04/06 15:20:45 ERROR shuffle.RetryingBlockFetcher: Exception while beginning fetch of 1 outstanding blocks java.io.IOException: Failed to connect to /172.16.115.136:42373 at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:287)
'2021년 > ERROR' 카테고리의 다른 글
No module named google (0) 2021.06.10 bigquery 에러 (0) 2021.06.02 Exception: Python in worker has different version 3.6 than that in driver 3.8 (0) 2021.04.06 [Hadoop Jps] namenode, datanode가 실행되지 않는다면? (0) 2021.04.05 conflicts with the name of an existing Python module and cannot be used as an app name. Please try another name (0) 2021.03.26