Apache Spark Certification Practice Test

Question: 1 / 400

Where do you keep a list of hostnames that run worker nodes in a Spark cluster?

conf/slaves

The correct answer is based on the configuration practices within Apache Spark. In a typical Spark setup, the cluster's worker node hostnames are specified in a file named `slaves`, which is located in the `conf` directory. This file serves as a central reference for the Spark master to locate and communicate with each worker node in the cluster. By listing the hostnames in the `slaves` file, the Spark master can efficiently assign tasks and distribute resources across the available worker nodes.

Other options do not serve the same purpose. For instance, the file `nodes` is not a standard configuration file recognized by Apache Spark for listing worker nodes. Similarly, `workers` may suggest a similar function but is not the officially recognized filename for this purpose. The `/etc/hosts` file on the other hand, is used for mapping hostnames to IP addresses on the local system but does not specifically pertain to listing worker nodes in a Spark cluster context. Therefore, it does not fulfill the operational requirement of defining the worker nodes.

Get further explanation with Examzify DeepDiveBeta

conf/nodes

bin/workers

etc/hosts

Next Question

Report this question

Subscribe

Get the latest from Examzify

You can unsubscribe at any time. Read our privacy policy