Dell+PowerProtect+DD+System+Upgrade+-+Participant+Guide

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 41

DELL POWERPROTECT

DD SYSTEM UPGRADE

PARTICIPANT GUIDE

PARTICIPANT GUIDE
Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 2


Table of Contents

Dell PowerProtect DD System Upgrade 4


About DD Operating System Upgrades 5
Discovering Why to Upgrade 5
DD Operating System Releases 6
Minimally Disruptive Upgrade (MDU) 8

Upgrading the PowerProtect DD System 9


Preparing for a System Upgrade 9
Performing a System Upgrade 11
System Upgrades Using PowerProtect DD Management Center 15
DDOS Upgrade Rollback 19
Performing a System Upgrade Using the Command-Line Interface 21
General PowerProtect DD Operating System Upgrade Instructions 22
Upgrade Preparation 23
PowerProtect DD System Health Check and Reboot 24
Upgrading the DD System Using the Command-Line Interface 30
PowerProtect DD System Upgrade Validation 32
Upgrade Considerations for HA Systems 33
Troubleshooting DDOS Upgrades 36
Troubleshooting Upgrades - Precheck 36
Troubleshooting Upgrades - Timeout 37
Troubleshooting Other Upgrade Issues 39

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 3


Upgrading the PowerProtect DD System

Dell PowerProtect DD System Upgrade

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 4


Upgrading the PowerProtect DD System

About DD Operating System Upgrades

Discovering Why to Upgrade

The DD Operating System (DDOS) is the


intelligence that powers the Dell PowerProtect
DD series appliances. When purchasing a new
DD series appliance, you can now consume
DDOS as a subscription, providing flexibility for
deployment while minimizing upfront costs.

The following are several reasons to upgrade


PowerProtect DD series appliances that are
running as a part of a data protection deployment:

• Upgrades to DDOS install operating system improvements. Upgrades


are not always essential, but Dell Technologies recommends that you
maintain PowerProtect DD appliances with the current versions of
DDOS. Using the most current DDOS ensures that you have access to
all the features and capabilities that your system has to offer.
• Stay up-to-date with the latest security updates. Updates enhance
existing features, patch security flaws, add new security features, fix
issues, and improve system performance.
• Upgrading DDOS ensures better compatibility with new data protection
systems and software. When you add newer PowerProtect DD
systems to your backup architecture, you must often update the DDOS
version to support hardware changes. For example, remote-battery,
non-volitile RAM (NVRAM), or newer model expansion shelves might
require a more current version of DDOS.
• Administrators who want to take advantage of all replication options
between source and destination devices should consider upgrading
their PowerProtect DD systems to the most current version of DDOS.
Replication pairs running earlier versions of DDOS can still support
replication with some limitations. For example, directory replication is
supported in DDOS 7.7.0.0 and earlier but not in versions after 7.7.0.0.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 5


Upgrading the PowerProtect DD System

Only DDOS 6.2 and later supports MTree and managed file replication.
Dell Technologies recommends that systems that are paired in a
replication configuration should have the same version of DDOS.
• Upgrading the DDOS ensures better compatibility with backup host
software. Administrators upgrading backup host software should
always check the minimum DDOS version that Dell Technologies
recommends for your backup software. Go to the Dell Support website
and view the current DDOS Release Notes to learn more about the
supported backup host software.
• You can correct unexpected system behavior by upgrading to a more
current DDOS. No software is free of flaws, and Dell Technologies
works continuously to improve the functionality of DDOS. Each version
release has complete release notes that identify issue fixes by number.

DD Operating System Releases

Dell Support Website Showing Available Releases for DD9500 Appliance

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 6


Upgrading the PowerProtect DD System

DDOS provides the agility, security, and reliability that enables the
platform to deliver scalable, high-speed, and cloud-enabled protection
storage for backup and archive.

DDOS functionality is constantly improving with each new version. Go to


the Dell Support website to locate and download available upgrade
packages for your PowerProtect DD products.

To upgrade the system, download an upgrade package from the Dell


Support website to a local system and then transfer it to the target system.
Through the DD System Manager UI you can view and manage up to five
upgrade packages on a PowerProtect DD system.

By upgrading to the latest release, you ensure that you are running the
version of DDOS with the highest reliability status.

Caution: Reverting to a previous DDOS version destroys


all data on the PowerProtect DD system.
There is no downgrade path to a previous version of
DDOS. The only method to revert to a previous DDOS
version is to destroy the file system and all the data that is
stored on that system.
If necessary, contact Dell Support to discuss system
backup and restoration options before you upgrade.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 7


Upgrading the PowerProtect DD System

Minimally Disruptive Upgrade (MDU)

The minimally disruptive upgrade (MDU)


feature lets you upgrade specific software
components or apply fixes to issues
without performing a system reboot. An
MDU is similar to the Linux atomic
upgrade. The MDU is made of stand-alone
component Red Hat Package Managers
(RPMs). These stand-alone components
come in smaller packages to facilitate
faster delivery to the system.

Not all software components qualify for an


MDU; you must upgrade some components as part of a regular DDOS
system software upgrade.

A DDOS software upgrade uses a large Red Hat Package Manager


(RPM) upgrade bundle, to perform upgrade actions for all of the
components of DDOS. An MDU uses smaller component bundles, which
upgrade specific software components individually.

When an administrator upgrades the system for a specific component, the


upgrade triggers an MDU.

Only those services that depend on the component that the MDU
upgrades are disrupted. The MDU feature can prevent significant
downtime while performing other operations during certain software
upgrades.

Go to: Go to the Dell Support website to determine if an


MDU is available for a specific DDOS issue.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 8


Upgrading the PowerProtect DD System

Upgrading the PowerProtect DD System

Preparing for a System Upgrade

Identify the currently running version of DDOS on the target system. In DD


System Manager, go to DASHBOARD > MAINTENANCE to determine the
current running DDOS version.

You can also check the currently running version using the command-line
interface command, system show version.

Once you verify the DDOS compatibility with the various software
applications and hardware devices that your protection system uses, you
can prepare to upgrade.

Preparing to Upgrade

Some versions of DDOS require intermediate updates using MDU updates


before upgrading to the target DDOS version. Go to the section titled
Preupdate Checklists and Compatibility Matrix in the target DDOS version
of Dell DDOS Release Notes. Confirm the correct MDU update process
for your system.

If you determine that you must update your system with an MDU, do the
following:
• Upload the MDU bundle mdu-update-bundle--
releases.rpm according to the version running on the system,
update mdu-bundle-x.x-releases.rpm. Upload the RPM to the
same location as a system RPM, /ddvar/releases/.
• Perform a system update with the MDU RPM, which should take a few
minutes.
• After you complete the MDU update, you can move on to complete the
update to the target version of DDOS.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 9


Upgrading the PowerProtect DD System

Pre-Upgrade Considerations

Before running a DDOS upgrade, consider the following:


• Dell Technologies recommends that you have at least 10% free space
available on the DD system before updating.
• If you are using NetWorker with Data Domain VTL, set the NetWorker
tape block size on the media server to 256 KB. 256 KB is safe for all
operating systems and drivers.
• If you are using NetWorker with Data Domain VTL, set NetWorker
device properties Target Sessions and Maximum Sessions to 1.
Changing these properties avoids low deduplication rates due to the
multiplexing of multiple backup streams.
• The upgrade cannot proceed if LUN numbers that are greater than 255
and initiators are present in the same group. The update proceeds
once you remove all the LUN numbers greater than 255 from the
identified groups or one or more initiators from the identified group.
• Resolve any active alerts on the PowerProtect DD system that you are
upgrading. View all the current alerts in DD System Manager select
Health > Alerts > Current Alerts.
• Reboot the PowerProtect DD system that you are upgrading.
Rebooting can resolve underlying or unreported issues.

− Confirm that the file system is enabled after the preupgrade reboot.
− Ensure that no new alerts appear after the preupgrade reboot.

Go to:Review the Preparing to Update section in Dell


DDOS Release Notes for your target version for any further
third-party application or configuration considerations before
proceeding. Release notes are available on the Dell Support
website.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 10


Upgrading the PowerProtect DD System

Deep Dive: Review the PowerProtect DD compatibility


Matrix before proceeding with the DDOS upgrade to ensure
product compatibility.

Important: Before performing any upgrade Dell


Technologies recommends that you read the release notes
for the DDOS system version you are upgrading to. The
release notes contain an upgrade section that provides all
the necessary information about how to perform the
upgrade, including prechecks and any preparatory steps
that you might need.

Performing a System Upgrade

Verify the compatibility of the target DDOS with the various software
applications, and hardware devices that are related to your system. When
you verify compatibility, you can prepare to upgrade.

Important: Read the section titled Prepare the system for


upgrade in the Dell DDOS Release Notes for the version of
DDOS you are upgrading to. Follow all of the steps in the
task list. Complete the upgrade pre-checks described in this
section approximately one week before the scheduled
DDOS upgrade to allow time to address any issues found
during the pre-checks.

Schedule at least 2 hours of downtime for a PowerProtect DD Operating


System (DDOS) upgrade. The time to complete the upgrade varies due to
the PowerProtect DD system size and the upgraded DDOS version. For
high-availability systems, you need extra time to perform a failover and

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 11


Upgrading the PowerProtect DD System

failback prior to initiating an upgrade. Depending on the PowerProtect DD


model you might need an additional hour or longer.

Download and Install the Upgrade Package

Locate and download the DDOS upgrade package do the following:


1. Go to Data Domain: DD OS Software Versions on the Dell Support
website and locate the DDOS version release that you want to upgrade
to.
2. Click Download.
A PDF document that contains an end user license agreement
appears.
3. Scroll to the bottom of the document and click I Accept to accept the
End User License Agreement and go to the READ ME FIRST page.
A PDF document that contains the READ ME FIRST page appears.
4. Click the link to the DDOS RPM file to start the download. When the
download is completed, confirm that the hash value for the file matches
the hash value on the download page.
5. Open the PowerProtect DD System Manager (DDSM) and log in as
sysadmin.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 12


Upgrading the PowerProtect DD System

6. In the MAINTENANCE > SYSTEM tab, select UPLOAD UPGRADE


PACKAGE and select your file. DDOS automatically stores the file in
the /ddvar/releases directory on the PowerProtect DD system.
7. In the MAINTENANCE > SYSTEM > VIEW CHECKSUM to confirm the
checksum of the upgrade file.

Upgrade Precheck

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 13


Upgrading the PowerProtect DD System

Begin the system upgrade. Go to Maintenance > System and perform the
UPGRADE PRECHECK.

In DD System Manager, click Upgrade Precheck in the Maintenance >


System > UPGRADE PRECHECK tab. Upgrade precheck is part of the
upgrade process which determines whether the system is in an
upgradable state.

The aim of the precheck is to detect potential problems early and cancel
the upgrade. If you perform the upgrade without a precheck, you might
place the system in an unusable state after an upgrade attempt.

Perform the Upgrade

To perform the upgrade from the DD System manager, ensure that the
system is idle. Confirm that all replication initialization and
resynchronization operations are complete and that no backup operations
are running. The alert show current CLI command is a useful
command to assess the readiness of the system.

To upgrade a system, do the following:


1. Perform a pre-upgrade reboot
2. Log out of the command-line interface (CLI).
3. Log in to the DD System Manager on the system you are upgrading.
4. Select Maintenance > System.
5. From the Upgrade Packages Available list, select the check box of
the image file to use for the upgrade.
6. Select PERFORM SYSTEM UPGRADE to begin the upgrade process.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 14


Upgrading the PowerProtect DD System

If you upgrade on a single node, non-high-availability system, a reboot


might take as many as 20 minutes or more. The reboot duration depends
on the features of your particular system. Do not manually reboot the
system during this time. During the reboot phase, the connection with the
system is lost.

When the reboot is complete, log back into the system.

System Upgrades Using PowerProtect DD


Management Center

You can use Dell PowerProtect DD Management Center (DDMC) to


upgrade one or more PowerProtect DD appliances registered on DDMC
instance.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 15


Upgrading the PowerProtect DD System

Overview

The Updates page allows you to upload a DD Operating System (DDOS)


update package to the DDMC inventory to schedule or perform a DDOS
update on the PowerProtect DD systems that DDMC manages.

Download the DDOS update package1 from the Dell Support website.

1Download the DDOS update package with the prefix x.x.rpm, for
example, 7.11.0.0-1035502.rpm.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 16


Upgrading the PowerProtect DD System

Managing System Update Packages

To upload a DDOS update package, follow the steps below:

1. Select Infrastructure > Updates.


2. Select the Packages tab.
3. In the Add Update Package dialog box, click BROWSE, and locate
the update package.
4. Click ADD.

When the update package uploads to the DDMC inventory, you can
update one or more systems.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 17


Upgrading the PowerProtect DD System

Scheduling a System Update

The systems you target for an upgrade must be in a reachable, active


state. If the system is not in an acceptable managed state, for example, if
DDMC reports the system is unreachable, suspended, or updating, the
update action is unavailable.

To schedule a system update, follow the steps below in DDMC:

1. Select Infrastructure > Updates.


2. Click the Configure Update button.
3. Set the Update Name, and select one of the options below, and the
click Next:
a. Download Package Only
▪ Allows for downloading an update package to the systems
without installation.
b. Install Update Only
▪ Allows for installation of previously downloaded update
packages on systems.
c. Download Package and Install Update
▪ Allows for configuring both download and installation of an
update package to selected systems.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 18


Upgrading the PowerProtect DD System

4. From the Select Systems page, select a system from the Available
Systems list and then click Add.
5. Click Next.
6. From the Select Package page, select the update package to apply to
the previously selected system or systems. Click Next.
− If the update package exists in the system, continue to the next
step.
7. The Schedule page depends on the option that is selected on step
number three:
− When you select Download Package Only,choose when to
download the package and then click Next.
− When you select Install Update Only, choose when to install the
update and then click Next.
− When you select Download Package and Install Update, choose
when to download the package and when to install the update and
then click Next.
8. From the Summary page, review the summary, and if applicable,
select Reboot before installation. Click Finish.

− Rebooting the system lets the update continue without any conflicts
from background processes.

DDOS Upgrade Rollback

In current DD Operating System (DDOS) versions, system engineers


improved the management and onboarding of PowerProtect DD series
appliances at scale. You can perform DDOS prechecks and scheduled
upgrades using PowerProtect DD Management Center (DDMC). When
the system detects an upgrade failure, it automatically rolls back to the
previous DDOS version. Auto rollback allows for continuous backup
operations even if an upgrade fails.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 19


Upgrading the PowerProtect DD System

When an administrator initiates a system upgrade, DDOS performs


prechecks on many of the system hardware and software parameters. The
DDOS checks the before initiating the backup operation:

• The partition size check verifies that the /ddr and /(root) partitions
are correct.
• The system performs a check on the redundant array of independent
disks (RAID) to ensure the core operating system disks are available
on the head unit.
• The precheck also determines whether the file system is enabled and
verifies that the numbers of MTrees and Virtual Tape Library (VTL)
pools are less than 100.
• An operations check ensures that the system is not performing file
system cleaning, cloud cleaning, and data movement operations.

If precheck determines a problem with any of these items, the upgrade


process quits without attempting an upgrade.
If the system performs an upgrade and encounters a failure with the
upgrade process, the upgrade stops. The system then automatically
returns to its pre-upgrade version.

You can view the status of an upgrade using the DDOS command,
system upgrade status. Log messages for the upgrade are
stored in /ddvar/log/debug/platform/upgrade-error.log
and /ddvar/log/ debug/platform/upgrade-info.log.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 20


Upgrading the PowerProtect DD System

You can review and capture the file content using the log view
debug/platform/upgrade-err.log log view
debug/platform/upgrade-info.log command.

Important: Rollback automatically occurs only when a


DDOS upgrade fails. DDOS offers no provision or support
for selectively downgrading to previous versions of DDOS.

Best Practice: If you are unable to determine the cause of


the rollback contact Dell Support for assistance.

Performing a System Upgrade Using the Command-


Line Interface

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 21


Upgrading the PowerProtect DD System

General PowerProtect DD Operating System Upgrade


Instructions

The main steps to perform a DDOS upgrade are:

1. The upgrade preparation includes choosing and uploading the DDOS


upgrade package to the system, reading the release notes, and
checking the compatibility guide to ensure that this DDOS version is
compatible with your backup environment.
2. The system health check includes running several commands using
the command-line interface (CLI) to ensure the PowerProtect DD
system is ready for the upgrade. A manual reboot is then required to
ensure that no hidden hardware issues are present.
3. The DDOS software upgrade performs an upgrade of all the
components of DDOS and reboots the system automatically.
4. The system upgrade validation includes running several commands
using the CLI to ensure the PowerProtect DD system upgrade
completed successfully.

Important: During the DDOS upgrades backups shall not


be running to avoid any upgrade failure.

Important: When upgrading from DDOS 7.6 or earlier to


DDOS 7.7 or later, if the system encounters an error that
causes the upgrade to fail, the system automatically rolls
back the upgrade to the previously installed working version.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 22


Upgrading the PowerProtect DD System

Upgrade Preparation

Obtain the PowerProtect DD Operating System (DDOS) upgrade package


from the Dell Support website. Review the release notes and the product
compatibility matrix, and upload the upgrade package to the system.

There are various methods to upload the DDOS upgrade package to the
PowerProtect DD System:

• Using the PowerProtect DD system Manager (DDSM) go to


Maintenance > System > UPLOAD UPGRADE PACKAGE.
• Creating a CIFS share to the /ddvar directory, then from a Windows
client mounts the /ddvar directory and uploads the DDOS upgrade
package to the releases directory.
• Creating an NFS export to the /ddvar directory, then from a UNIX or
LINUX client mounts the /ddvar directory and uploads the DDOS
upgrade package to the releases directory.
• You can use the scp <rpm file name> sysadmin@<hostname
or IP>:/ddr/var/releases/ command from a Windows, LINUX,
or UNIX terminal to copy the upgrade package to the
/ddr/var/releases directory of the PowerProtect DD system.

− The rpm file name is the DDOS upgrade file, for instance
7.13.0.0-1076200.rpm.

− The hostname or IP is the PowerProtect DD system hostname


or IP address.
• Using a USB drive, create a /upgrade directory and copy the upgrade
package there.

− The system upgrade precheck file evaluates whether the


operating system can be upgraded to the version in the specified
file. The system searches for the file in the /ddvar/releases
directory and on a USB drive. The precheck does not upgrade or
modify the system, and it does not impact system performance.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 23


Upgrading the PowerProtect DD System

Important: There is a 30-minute time limit for uploading an


RPM package for the PowerProtect DD system upgrade
using DDSM for security reasons. For slow connections
from the client machine to DDSM, upload the upgrade
package using SCP from a Windows, LINUX, or UNIX
terminal.

Deep Dive: Review the PowerProtect DD compatibility


matrix before proceeding with the DDOS upgrade to ensure
product compatibility.

Deep Dive: For more information about preparing the


system for an upgrade, see Upgrading the System chapter
of the DDOS Administration Guide from the Dell Support
website.

PowerProtect DD System Health Check and Reboot

To avoid an upgrade failure, complete the upgrade prechecks before a


DDOS upgrade to allow time to address any issues found during the
prechecks.

System Configuration Details

Use the commands below to gather the system configuration details:

Step Description Command

1 Verify the system serial number. system show serialno

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 24


Upgrading the PowerProtect DD System

2 Verify the hardware model number system show modelno


of the system.

3 Verify the hostname of the system. net show hostname

4 Verify the system network net show config


configuration. net show hardware

5 Verify all network interfaces. net show settings

6 Verify the list of DNS servers that net show dns


the system uses.

7 Verify the system replication replication show


configuration. config all

System Health Check

Use the commands below to perform a system health check and ensure
that there are not alerts on the system:

Ste Description Command


p

1 Check for alert show current


current alerts.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 25


Upgrading the PowerProtect DD System

2 Verify the disk disk show state


states. Do not
perform the
upgrade if the
system is low
on spares or
has disks
showing
absent, failed,
or
reconstructing
.

3 Verify the disk disk show reliability-data


reliability.

4 Verify the enclosure show all


enclosure
status.

5 Verify that the enclosure show topology


enclosure
topology is
correct.

6 Verify that the system show hardware


device port
mapping is
correct.

7 Verify the link system show ports


speed for
connected
ports.

8 Verify the filesys status


status of the
file system.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 26


Upgrading the PowerProtect DD System

9 Verify the replication status


replication
status.

10 Verify the VTL vtl status


library status.

11 Verify if any system show stats


backup and
restore
activity is in
progress, and
if so, stop it.

12 Verify the filesys show space


space
available to
and used by
file system
resources.
Values are
reported in
gibibytes
(GiB). Ensure
that there is at
least 10% of
free space
available on
the system.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 27


Upgrading the PowerProtect DD System

13 Run an autosupport send <your_email_address>


autosupport
report before
performing
the DDOS
upgrade to
determine if
the system
reports errors
that must be
resolved
before the
upgrade.

14 If SMTP is not autosupport show report


working,
capture the
autosupport
report results.

15 If the log view


autosupport debug/platform/debug/platform/kern.in
report fo
indicates any
issues with
the system,
check the
kern.inf
log.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 28


Upgrading the PowerProtect DD System

Reboot the System

Reboot the system to verify that the hardware is in a clean state.

Reboot the PowerProtect DD system using the system reboot


command.

After the system reboots, run the following commands:

• alert show current to ensure that no new alerts have been


generated after the reboot.
• filesys status to ensure that the file system is enabled.

Resolve any issues discovered during the reboot.

Important: If during the prechecks any issue is found,


contact Dell Support to inspect the system and address all
the issues before you perform the upgrade.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 29


Upgrading the PowerProtect DD System

Deep Dive: For more information about the PowerProtect


DD system commands, see the DDOS Command
Reference Guide from the Dell Support website.

Upgrading the DD System Using the Command-Line


Interface

Starting the DDOS Upgrade

To upgrade the PowerProtect DD Operating System (DDOS), follow the


steps below:

1. Open an SSH session to the PowerProtect DD system.


2. Run the system package list <rpm file name> command to
confirm the upgrade package file is in the /ddvar/releases
directory.
3. Run the system upgrade precheck <rpm file name>
command prior to upgrading DDOS. The command evaluates whether
the operating system can upgrade to the version in the specified file.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 30


Upgrading the PowerProtect DD System

The precheck does not upgrade or modify the system, and it does not
impact system performance. The precheck evaluates a set of
parameters and indicates whether or not there are any issues that
might prevent an upgrade.
4. Upgrade the system using the system upgrade start <rpm
file name> command.
a. The rpm file name is the DDOS upgrade file, for instance
7.13.0.0-1076200.rpm.

5. Type Yes and press ENTER to continue.

The upgrade starts with a precheck of the system to determine if the


upgrade can continue.

If the precheck fails, the command terminates without modifying the


system.

System Reboot

When the upgrade is nearly complete, the system shuts down the file
system and reboots.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 31


Upgrading the PowerProtect DD System

When an upgrade is in progress, you can monitor the upgrade from


another SSH session using the commands below:

• To see the upgrade progress use the system upgrade status


command.
• To continually see the upgrade process, use the system upgrade
watch command. To terminate the command press CTRL-C.

• If you are performing upgrades while onsite, establish a serial


connection to view additional system output during the reboot process.

PowerProtect DD System Upgrade Validation

Use the commands below to perform a system health check and validate
that the DDOS upgrade is completed and the system is up and running:

Step Description Command

1 Verify that the PowerProtect DD system system upgrade


upgrade completed successfully. status

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 32


Upgrading the PowerProtect DD System

2 Verify the history of system upgrades. system upgrade


history

3 Verify the PowerProtect DD Operating system show


System (DDOS) version. version

4 Check for current alerts. alerts show


current

5 Verify the status of the file system. filesys status

6 Verify the system serial number. system show


serialno

Deep Dive: For a more information about the PowerProtect


DD system commands, see the DDOS Command
Reference Guide from the Dell Support website.

Upgrade Considerations for HA Systems

When upgrading an HA system, upload the upgrade RPM package to the


active node.

Steps Before Initiating the HA System Upgrade

1. Run the ha status command to verify the HA system state.


a. The HA system must be in a highly available state, with both nodes
online before performing the DDOS upgrade.

2. Reboot the standby node node 1 using the system reboot


command.
3. After the standby node reboot, run the ha status command to verify
the HA system status displays as highly available.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 33


Upgrading the PowerProtect DD System

4. Run the ha failover command to initiate a failover from the active


node to the standby node.

5. Run the ha status command to verify that the node 1 is the active

node and node 0 is the standby node.

6. Reboot the standby node node 0 using the system reboot


command.
7. After the standby node reboot, run the ha status command to verify
the HA system status displays as highly available.

8. Run the ha failover command to initiate a failover from the active


node to the standby node.

9. Run the ha status command to verify that the node 0 is the active

node and node 1 is the standby node.

Initiate Upgrade on the Active Node

DDOS automatically recognizes the HA system and performs the upgrade


procedure on both nodes.

The HA upgrade runs in the following sequence:

1. The standby node is upgraded first and then reboots.


2. After the reboot, the HA system initiates a failover and the standby
node takes over as the active node.
3. The original active node is upgraded, reboots, and remains as the
standby node.

After the upgrade procedure is complete, run the ha status command to


verify that the system is in a highly available state and both nodes are
online.

Optionally run the ha failover command to return the nodes to the


roles they were in before the upgrade.

If for some reason a rolling upgrade fails unexpectedly, consider a local


upgrade as a fix method in this situation.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 34


Upgrading the PowerProtect DD System

A local upgrade functions as follows:

1. Upload the upgrade package on both active and standby nodes.


2. Run a precheck on the upgrade package on both active and standby
nodes.
3. Take the standby node offline.
4. Perform the upgrade on the standby node. After the upgrade finishes,
the standby node reboots.
5. Perform the local upgrade on the active node. After the upgrade
finishes, the active node reboots.
6. After the active node upgrade finishes, the HA system status is still
degraded.
7. Run the ha online command to make the standby node online.
8. The standby node reboots and rejoins the cluster, and the HA status
becomes highly available.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 35


Troubleshooting DDOS Upgrades

Troubleshooting Upgrades - Precheck

All PowerProtect DD upgrades are considered disruptive to the system.


Upgrades take the system offline. Plan a downtime window for upgrades
as the system reboots as part of the upgrade process. Confirm that no
backups, restores, replications, or cleaning operations run during the
upgrade window.

If any operations are active during an upgrade precheck, you could


receive an error. The following is an example alert that might appear:

To troubleshoot, you can check the infra.log file to confirm whether


file system cleaning is the cause of a DDOS upgrade precheck. The
command to view the file is log view debug/platform/infra.log.
Review the infra.log file and confirm possible problems that the
system logged during the time you ran the upgrade precheck:

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 36


You can work around this issue or any issue where a file system operation
runs during an upgrade precheck. Stop file system cleaning or wait for
cleaning to complete before retrying the upgrade precheck.

Use the following command to manage file system cleaning: # filesys


clean <status | watch | stop>

Troubleshooting Upgrades - Timeout

In some rare cases, a DD Operating System (DDOS) upgrade may not


report a status and timeout. A DDOS upgrade may fail due to a timeout in
the upgrade process. A timeout during an upgrade gives no indication of
the state of the system or the steps to address the upgrade failure.

Upgrade Phases

The DDOS upgrade process consists of two major phases:


• Stage 1 is the pre-reboot stage from starting the upgrade to the point
where the system is automatically rebooted into the new version of
DDOS.
• Stage 2 is the post-reboot stage that follows the automated reboot
during which the new operating system upgrades volume, data,
registry, and other system elements. Stage two concludes after the DD
File System restarts.

Upgrade Timeout

A short message service (SMS) task controls the pre-reboot phase of the
DDOS upgrade. The task regularly examines the progress of the upgrade
to confirm that the upgrade proceeds as it should. In some circumstances,
the pre-reboot stage of the upgrade may take longer than the timeout set
by the SMS task. At the timeout, usually 20 minutes, the SMS task
terminates without warning. The system does not attempt any further

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 37


upgrade steps. The appliance does not automatically reboot into the new
version of DDOS.

Troubleshooting the Timeout

If you suspect that the upgrade process stalls or stops due to a timeout,
you can review SMS logs during the timeframe of the upgrade. The SMS
logs are in /ddr/var/log/debug/sm/sms.info.

The following is a sample output of sms.info indicating the timeout error:

Use the log view debug/platform/infra.log command to view


which upgrade step took the most time. The infra.log can also display
which step the upgrade process ran before the SMS task timed out.
Sometimes all the required steps may have been completed except for the
physical reboot. If all the steps are completed, all that the system needs is
a manual reboot. In other cases, only some upgrade tasks may have been
completed. If the upgrade partially completes, you should avoid rebooting
because the appliance might be in an unstable state.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 38


Resolution

Several reasons can cause the pre-reboot phase of an upgrade to exceed


the set time and end the SMS task. Some common reasons are:
• Usually, you can rerun the DDOS upgrade. Potentially long-running
tasks such as an index merge completed during the initial upgrade
attempt. The merge is either not required to run again or is quicker
than during the initial upgrade attempt.
• Consider running a file system cleaning operation to reduce the
amount of work that the index merge process must complete during an
upgrade. Confirm that the operation finishes before rerunning the
upgrade.
• Quiesce operations on the system: Stop running any operations that
are running in the background. Background operations include backup
and restore, file system cleaning, and any managed file replication
operations before rerunning the upgrade.
• If the DD File System (DDFS) slows or times out during shutdown as
part of the upgrade, you might proactively disable DDFS before
rerunning the upgrade.
• Examine and tail infra.log to determine the progress of the upgrade
and identify which operations take excessive time to complete.

Troubleshooting Other Upgrade Issues

DDOS upgrade performs certain tests before a system upgrade. If a test


fails, the upgrade could fail. The following circumstances could cause the
upgrade to fail:

Running Tasks

The following circumstances could cause the upgrade to fail when some
tasks are running:

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 39


• If a replication initialization is in progress.
• The file system did not shut down cleanly, resulting in a core dump.
• A previous upgrade did not complete successfully.

Configuration Issues

The following circumstances could cause the upgrade to fail when the
system is configured incorrectly:

• NFS mount points are manually created under root or unknown.


• Inspection of digests and signatures that are contained in .RPM file
indicates that the signature is not valid. The valid signatures are SHA1
or MD5.
• The Open Storage Technology (OST) or Virtual Tape Library (VTL)
versions are incompatible with the new DDOS version.
• The file system is not enabled or cannot be enabled automatically.
• The original version of DDOS is not compatible with the target version.

Space Issues

The following space issues could cause the upgrade to fail:

• If storage is functionally deficient, such as when an enclosure is


missing.
• If the /(root) or /ddr partition is full with log files, core dumps, and
so on.
• If the available space for the file system is insufficient.
• If the available space for VTL is not sufficient.
• If the number of MTrees or VTL pools exceeds 100.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 40


Deep Dive: Go to the current Dell EMC DDOS
Administration Guide on the Dell Support website for
detailed information about system upgrades.

Deep Dive: Search for upgrade-related knowledge base


articles by searching on the Dell Support website for
detailed information.

Dell PowerProtect DD System Upgrade

© Copyright 2023 Dell Inc Page 41

You might also like