代做java | oop代写 | project | 代写hadoop | unity | lab – COMP4057 Distributed and Cloud Computing

2022-08-17

COMP4057 Distributed and Cloud Computing

代做java | oop代写 | project | 代写hadoop | unity | lab – 这是一个关于hadoop的题目, 主要考察了关于hadoop的内容,是一个比较经典的题目, 是有一定代表意义的java/oop/hadoop/unity等代写方向, 该题目是值得借鉴的lab代写的题目

OOP代写 代写oop

lab 1A Introduction to Programming on Cluster

Using IntelliJ

Introduction

The purpose of this document is to setup simple environments (IntelliJ) to write programs that can run on the big-data cluster or on your PC. Note that although you use IntelliJ (IDE, integrated development environment) to build your project, this tutorial still requires you to log in one of the machines in the cluster to submit a job.

Downloading IntelliJ Ultimate

Before downloading IntelliJ, you need to apply for a free student pack through the following
URL: https://www.jetbrains.com/buy/classroom/?fromMenu. You should use your HKBU
email address. After finishing the application form, you will receive an email from JetBrains
team, and remember to confirm the request.

 Click it

Then you may download theComm unity version of IntelliJ for your computer through the following URL:https://www.jetbrains.com/idea/download/ You may downloadUltimate version for a trial.

Configuring IntelliJ [Utimate Version only]

Configuring the deployment path
  1. After installing IntelliJ, launch it and create a new java project. Then, click the menu Tools -> Deployment -> Configuration (to setup a connection with your server)
  2. In the Deployment dialog window, click +and choose SFTP. Then, type research server for the name. Make sure you have already setup port forwarding if you use this at home.
  3. In the Connection tab, fill the information as follows:
Host csrXX.comp.hkbu.edu.hk
(where XX is a number in the range from 51 to 59)
Port 22
User name and
Password

Your username and password for signing in the research
machines
  1. In the Mapping tab, map the local path to a path in your research machine, and then chooseok.
  2. Then, enable automatic upload by clicking the menu Tools -> Deployment -> Automatic upload.
Remark: automatic upload can be useful only if you connect to the HKBUs network

Adding Hadoop

Next, we import the required packages from the Had oop library to IntelliJ. After completing the following steps, IntelliJ can provide suggestions for your coding.

  1. Download the hadoop library from Moodle (hadoop.rar).
  2. Extract the files from the compressed file.
  3. Launch IntelliJ and open the project created previously. Then, click the menuFile -> Project Structure. In the Project Structure dialog window, clickModules -> Dependencies -> + -> JARs or directories…
  1. In the pop-up window, locate and expand the hadoop directory, and select all its subdirectories. Then, clickOKto attach these files to your project.
Then, when you are programming and using the packages of Hadoop, IntelliJ will give
you suggestions.
  1. Connect to the node of the cluster. If you are connecting to the campus Intranet, you can use IntelliJ [Ultimate version only] to connect to the cluster directly. Open IntelliJ, click the menuTools -> Start SSH session and select the server you create before, you will see the session start at the bottom of the window.
  2. After coding, click the menuFile -> Project Structure.
  3. In the pop-up window, clickArtifacts -> + -> JAR -> From modules with dependencies.
  4. Fill in the output directory, the .jar file will be generated in this path then.
  1. Click the menuBuild -> Build Artifacts. If your program is correct, a jar file will be built in the path you fill in before.

10.After you finished the compilation and packaging, you should include the package in the class path in order to let Hadoop to use it:

$ export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/comp/$USER/src/*