HTCondorGPU - Revision history

Hanbi: /* HTCondor에서 GPU 사용하기 */

2021-05-12T02:22:40Z

HTCondor에서 GPU 사용하기

← Older revision		Revision as of 02:22, 12 May 2021
Line 147:		Line 147:
	위의 예제들은 아래 문서들에서 참고했습니다. 좋은 내용이 많으니 이용에 참고하시기 바랍니다.		위의 예제들은 아래 문서들에서 참고했습니다. 좋은 내용이 많으니 이용에 참고하시기 바랍니다.

	- https://batchdocs.web.cern.ch/tutorial/exercise10.html~~<br>~~		* https://batchdocs.web.cern.ch/tutorial/exercise10.html
	- https://www.tensorflow.org/tutorials/quickstart/beginner~~<br>~~		* https://www.tensorflow.org/tutorials/quickstart/beginner
	- https://sylabs.io/guides/3.6/user-guide/singularity_and_docker.html~~<br>~~		* https://sylabs.io/guides/3.6/user-guide/singularity_and_docker.html
	- https://sylabs.io/guides/3.6/user-guide/gpu.html		* https://sylabs.io/guides/3.6/user-guide/gpu.html

2021-05-12T02:15:38Z

참고자료

← Older revision		Revision as of 02:15, 12 May 2021
Line 145:		Line 145:
	=== 참고자료 ===		=== 참고자료 ===

	위의 예제들은 아래 문서들에서 참고했습니다.		위의 예제들은 아래 문서들에서 참고했습니다. 좋은 내용이 많으니 이용에 참고하시기 바랍니다.

	- https://batchdocs.web.cern.ch/tutorial/exercise10.html<br>		- https://batchdocs.web.cern.ch/tutorial/exercise10.html<br>

2021-05-12T02:15:02Z

참고자료

← Older revision		Revision as of 02:15, 12 May 2021
Line 144:		Line 144:

	=== 참고자료 ===		=== 참고자료 ===

			위의 예제들은 아래 문서들에서 참고했습니다.

			- https://batchdocs.web.cern.ch/tutorial/exercise10.html<br>
			- https://www.tensorflow.org/tutorials/quickstart/beginner<br>
			- https://sylabs.io/guides/3.6/user-guide/singularity_and_docker.html<br>
			- https://sylabs.io/guides/3.6/user-guide/gpu.html

2021-05-12T02:13:02Z

Ex 5) Singularity & TensorFlow - docker image

← Older revision		Revision as of 02:13, 12 May 2021
Line 127:		Line 127:

	=== Ex 5) Singularity & TensorFlow - docker image ===		=== Ex 5) Singularity & TensorFlow - docker image ===

			sing-docker.sub

			executable = sing.sh
			arguments = $(ClusterId)$(ProcId)
			output = sing.$(ClusterId).$(ProcId).out
			error = sing.$(ClusterId).$(ProcId).err
			log = sing.$(ClusterId).log
			should_transfer_files = YES
			when_to_transfer_output = ON_EXIT
			transfer_input_files = tf_ex.py
			request_GPUs = 1
			request_CPUs = 1
			+SingularityImage = "docker://tensorflow/tensorflow:latest-gpu"
			queue

	=== 참고자료 ===		=== 참고자료 ===

2021-05-12T02:12:28Z

Ex 4) Singularity & TensorFlow - local image

@@ Line 110: / Line 110: @@
 === Ex 4) Singularity & TensorFlow - local image  ===
 === Ex 5) Singularity & TensorFlow - docker image  ===
 === 참고자료 ===

2021-05-12T02:11:29Z

Ex 3) Singularity & TensorFlow - cvmfs image

← Older revision		Revision as of 02:11, 12 May 2021
Line 89:		Line 89:

	=== Ex 3) Singularity & TensorFlow - cvmfs image ===		=== Ex 3) Singularity & TensorFlow - cvmfs image ===

			sing.sh

			#!/bin/bash
			python3.6 tf_ex.py

			sing.sub

			arguments = $(ClusterId)$(ProcId)
			output = sing.$(ClusterId).$(ProcId).out
			error = sing.$(ClusterId).$(ProcId).err
			log = sing.$(ClusterId).log
			should_transfer_files = YES
			when_to_transfer_output = ON_EXIT
			transfer_input_files = tf_ex.py
			request_GPUs = 1
			request_CPUs = 1
			+SingularityImage = "/cvmfs/singularity.opensciencegrid.org/opensciencegrid/tensorflow-gpu:latest"
			queue

	=== Ex 4) Singularity & TensorFlow - local image ===		=== Ex 4) Singularity & TensorFlow - local image ===

2021-05-12T02:09:59Z

Ex 2) TensorFlow

@@ Line 66: / Line 66: @@
   model.fit(x_train, y_train, epochs=5)
   model.evaluate(x_test,  y_test, verbose=2)
 === Ex 3) Singularity & TensorFlow - cvmfs image  ===

2021-05-12T02:07:30Z

HTCondor에서 GPU 사용하기

← Older revision		Revision as of 02:07, 12 May 2021
Line 43:		Line 43:
	request_CPUs = 1		request_CPUs = 1
	queue		queue

			=== Ex 2) TensorFlow ===

			tf_ex.py

			import tensorflow as tf

			mnist = tf.keras.datasets.mnist

			(x_train, y_train), (x_test, y_test) = mnist.load_data()
			x_train, x_test = x_train / 255.0, x_test / 255.0

			model = tf.keras.models.Sequential([
			tf.keras.layers.Flatten(input_shape=(28, 28)),
			tf.keras.layers.Dense(128, activation='relu'),
			tf.keras.layers.Dropout(0.2),
			tf.keras.layers.Dense(10, activation='softmax')
			])

			model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])

			model.fit(x_train, y_train, epochs=5)
			model.evaluate(x_test, y_test, verbose=2)

			=== Ex 3) Singularity & TensorFlow - cvmfs image ===

			=== Ex 4) Singularity & TensorFlow - local image ===

			=== Ex 5) Singularity & TensorFlow - docker image ===

			=== 참고자료 ===

2021-05-12T02:03:47Z

HTCondor 에서 GPU 사용하기

2021-05-12T02:03:15Z

Ex 1) Matrix

← Older revision		Revision as of 02:03, 12 May 2021
Line 21:		Line 21:
	duration = timer() - start		duration = timer() - start
	print(duration)		print(duration)

			matrix.sh

			#!/bin/bash
			python3.6 -m virtualenv myvenv
			source myvenv/bin/activate
			pip3 install numba
			python3.6 matrix.py

			matrix.sub

			executable = matrix.sh
			arguments = $(ClusterId)$(ProcId)
			output = matrix.$(ClusterId).$(ProcId).out
			error = matrix.$(ClusterId).$(ProcId).err
			log = matrix.$(ClusterId).log
			should_transfer_files = YES
			transfer_input_files = matrix.py
			when_to_transfer_output = ON_EXIT
			request_GPUs = 1
			request_CPUs = 1
			queue