<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=2877026&amp;fmt=gif">

Installation guide for upgrading CDH and Cloudera Manager

By Akash Patel - October 17, 2019

Upgrading Cloudera Distribution Hadoop and Cloudera Manager from 5.7 to 5.16 in a Docker container


1. Docker Installed
2. Cloudera Image container running

Follow this link if the pre-requisites are not met.

To use Spark2 in Cloudera, we require CDH 5.9 and above, and for CDH 5.9 you will require a compatible version of Cloudera Manager.

  1. Upgrading Cloudera Manager (CM) version from 5.7 → 5.16.x.

  2. Upgrading CDH (Cloudera Distribution Hadoop) version from 5.7 TO 5.16.x.

Upgrading Cloudera Manager (CM) version

Let’s first check the current version of Cloudera Manager. Go to Home → Support → About.

Open up a Docker Bash terminal and go to the directory /etc/yum.repo.d/.

  cd /etc/yum.repo.d/

List of configuration files available in the /etc/yum.repo.d/ Directory.

  ls -al

Open the “cloudera-manager.repo” file using the vi editor. To check the baseurl source available for Cloudera Manager, we can use an archive link as well as a local copy of the Cloudera Manager.

In this case, we have to use the source which is available at “archive.cloudera.com”.

We have used given base URL
Cloudera — CM5 version.
Operating System — Redhat 6 version.

vi cloudera-manager.repo

These are the details available in the cloudera-manager.repo.


Clean all repositories available in the Cloudera

yum clean all
yum makecache

Now, let’s stop all running Cloudera Services.

Stopping Cluster ServicesStopping Cluster Services

Stopping Cluster Services CompletedStopping Cluster Services Completed

Now, we are going to stop Cloudera Manager Services

Stopping Cloudera Manager ServicesStopping Cloudera Manager Services

Stopping Cloudera Manager Services CompletedStopping Cloudera Manager Services Completed

Now, let’s stop Cloudera Manager Services: Cloudera-scm-server and Cloduera-scm-agent.

service cloudera-scm-server stop
service cloudera-scm-agent stop
cloudera-scm-agent cloudera-scm-server1

Let’s list the Cloudera services.

yum list all | grep cloudera
grep-cloudera grep-cloudera1 grep-cloudera2 grep-cloudera3

Let’s update Cloudera-manager-agent, Cloudera-manager-deamons and cloudera-manager-server.

yum -y upgrade cloudera-manager-agent cloudera-manager-daemons
cloudera-server cloudera-server1

Check the status of Cloudera Manager Services, and start them.

service cloudera-scm-agent status
service cloudera-scm-agent start
service cloudera-scm-server status
service cloudera-scm-server start

Now go to your browser, and openhttp://localhost:7180/cmf/login
1) Admin Credentials: admin/admin
2) Other Credentials: cloudera/cloudera

Cloudera Manager Login PageCloudera Manager Login Page

Since we update repositories, after login we will be landing at Upgrade Cloudera Manager. Click on Run Host Inspector.

Running Upgrade Cloudera ManagerRunning Upgrade Cloudera Manager

Completed Upgrade Cloudera ManagerCompleted Upgrade Cloudera Manager

Click Start Cloudera Management Service

Cloudera Management ServiceReview & Start Cloudera Management Services

Click Continue

Click ContinueReview & Start Cloudera Management Services

Click Continue

Start Cloudera Manager ServicesStart Cloudera Manager Services

Click Finish

Now, our Cloudera Manager Services are running using CM 5 version.

Upgrade Cloudera Manager ScreenUpgrade Cloudera Manager Screen

Click on Home Page.

You might see some services are in the stale configuration and require a restart.

Cloudera Manager Home Page after UpgradeCloudera Manager Home Page after Upgrade

(Optional) If you want, you could deploy new configurations. Click on the stale configuration button placed near each service and follow the steps.

Deploying New Client ConfigurationsDeploying New Client Configurations

Validate Cloudera Manager (CM) Upgrade

Go to Home → Support → About.

Cloudera Manager Home Page Support DropdownCloudera Manager Home Page Support Dropdown

Check, we are running Cloudera Express 5.16.2.

About Dialog BoxAbout Dialog Box

Start all Cloudera Hadoop services: Click Drop Down Icon → Start

Starting Hadoop ServicesStarting Hadoop Services

Cloudera Manager Home Page All Cluster Services RunningCloudera Manager Home Page All Cluster Services Running

Upgrading CDH (Cloudera Distribution Hadoop)

Go to Home → Hosts → Parcels

Cloudera Manager Hosts DropdownCloudera Manager Hosts Dropdown

List of parcels that are added by default in the configuration using URL. Cloudera CDH URL is added to the configuration. That’s why we are seeing CDH 5 in Parcel Name Column.

Note : These are not installed, you can instal them as per your requirement.

Available ParcelsAvailable Parcels

Click the Configuration button on top right

Parcel ReposParcel Repos

Now, let’s replace that “http://archive.cloudera.com/cdh5/parcels/5.7.0/” with “http://archive.cloudera.com/cdh5/parcels/5/”, as we want to upgrade CDH with the latest version of 5.x.x.

Parcel Repos with new entry addedParcel Repos with new entry added

You will see this CDH 5 with version 5.16.x. It will take a few seconds if you can’t find the correct version in the Parcels list. Click on Check for New Parcels.

New Parcel AvailableNew Parcel Available

Click on Download.

Note : It will take some time to download based on your internet speed.


Once it has downloaded, click on Distribute.

Parcel DownloadedParcel Downloaded

Now, click on Upgrade to enable a new CDH version.

Parcel DistributedParcel Distributed

Upgrading from 5.7.0 to 5.16.2 (Latest version)

Follow Checkpoints carefully.

upgrade cdh upgrade cdh1 parcel compat host inspector host inspector pass database backup cdh upgrade

This process takes a while…

cdh process

Now, our CDH 5 is activated through the parcel.


Now, let’s go to the home page. On the left sidebar, near the cluster name (Cloudera Quickstart..), you can find the currently running version.

cloudera manager

Congratulations!! We have successfully upgraded the Cloudera Manager (CM) and Cloudera Distribution Hadoop (CDH).

Also, check out our blog about how to install Spark2 in Cloudera Cluster through Docker here. To get the best data engineering solutions for your business, reach out to us at Clairvoyant.

Follow Clairvoyant to get more updates about data engineering.

Akash Patel

Tags: Data Engineering

Fill in your Details