Advertisements
Skip to content
November 28, 2015 / Shivananda Rao P

Fast-Start Failover in Oracle 11g

Fast Start Failover is a feature available through oracle dataguard broker which performs an automatic failover to the chosen standby database in case of a primary database crash.

 

FSFO involves an OBSERVER component which runs on a different machine than that of the primary and the standby database. The observer hardly consumes any resource and requires only a client or database software installed in it’s site and TNS connectivity established to the primary and the standby database. As the name suggests, the observer monitors the availability of the primary database and initiates a fast start failover when the primary database loses the connectivity with the observer and the chosen standby database target.

 

Before we move with configuring FSFO, there are some required FSFO properties that we need to know of. These include:

 

FastStartFailoverLaglimit: With FSFO, we can use only MaxAvailability or MaxPerformance mode. When in MaxPerformance mode, this property specifies the maximum amount of data loss that is permissible. This value is in “SECONDS” and the default value is 30 seconds. When in MaxAvailability mode, FSFO ensures that there is no data loss.

 

FastStartFailoverThreshold: This property specifies the number of seconds that the observer and the target standby database will wait before initiating the failover. By default, the value of this property is 30 seconds.

 

FastStartFailoverPmyShutdown: If this property is set to true and the primary database is stalled for more than FastStartFailoverThreshold seconds, then the primary database will shutdown.

 

FastStartFailoverTarget: This property specifies the db_unique_name of the database which would be the target database of the fast start failover.

 

FastStartFailoverAutoReinstate: If set to true, then the former primary database will be reinstated after the fast-start failover.

 

On another note, Fast-Start-Failover has a few restrictions:

 

1. You cannot change the protection mode of the dataguard broker configuration and nor can you change the log shipping mode (logxptmode) of the primary and the target standby database.

 

2. You cannot perform a switchover or a failover to a standby database which is not the “fast-start-failover target”.

 

3. The broker configuration cannot be removed if FSFO is eanbled and nor can the target standby database be deleted.

 

4. FSFO is not possible if the primary database was shutdown without the “abort” option or if the observer is not running.

 

Now let’s move on with the steps involved in enabling FSFO. This post assumes that a broker configuration already exists it’s creation is not outlined here.

The steps required to configure dataguard broker can be referred here https://shivanandarao-oracle.com/2013/07/10/data-guard-broker-configuration/

The configuration name used here is “dgtest” and uses the MaxAvailability mode.
Environment:

Primary Site:


DB Name            : srprim
DB Unique Name     : srprim
Connect Identifier : srprim
Hostname           : ora1-1

Standby Site:

DB Name            : srprim
DB Unique Name     : srpstb
Connect Identifier : srpstb
Hostname           : ora1-2

Observer Site:

Hostname : ora1-3

The configuration is enabled and it’s status is as below.

 

DGMGRL> show configuration;

Configuration - dgtest

  Protection Mode: MaxAvailability
  Databases:
    srprim - Primary database
    srpstb - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS

Make sure that both, primary and the standby databases have the flashback and FRA featured enabled which forms a pre-requisite for FSFO.
Now, let’s try to enable to “FSFO”.

 

DGMGRL> ENABLE FAST_START FAILOVER;
Error: ORA-16651: requirements not met for enabling fast-start failover

Failed.

This of-course fails. Reason — there are certain pre-requiste properties mentioned earlier which needs to be set.
Set the FastStartFailoverTarget to the target database (srpstb) for the primary database (srprim)

 

DGMGRL> edit database srprim set property 'FastStartFailoverTarget'='srpstb';
Property "FastStartFailoverTarget" updated
DGMGRL>

 

Similarly, set the FastStartFailoverTarget to the target database (srprim) for the database srpstb. This will be used if srpstb starts behaving as a primary database and srprim as a standby after the role transition.

 

DGMGRL> edit database srpstb set property 'FastStartFailoverTarget'='srprim';
Property "FastStartFailoverTarget" updated
DGMGRL>

Verfiy the property set above.

 

DGMGRL> show database srprim 'FastStartFailoverTarget';
  FastStartFailoverTarget = 'srpstb'

Now set the property FastStartFailoverThreshold to 60 seconds which will be time in seconds that the observer will wait before initiating the failover.

 

  
DGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold = 60;
Property "faststartfailoverthreshold" updated

Observer process is a continuous process and does not return the prompt at the DGMGRL session until you stop the observer from another DGMGRL session.
Due to this, it’s preferred to run the observer in background using the “nohup”.
In order to run the observer in the background, just connect to the broker configuration from the observer site and run the “start observer” command.

 

[oracle@ora1-3 ~]$ nohup dgmgrl sys/oracle@srprim "start observer" &

If willing to run in the foreground, then connect to the broker configuration from the observer site and run the “start observer” command to start the observer.

 

[oracle@ora1-3 ~]$ dgmgrl sys/oracle@srprim
DGMGRL for Linux: Version 11.2.0.2.0 - 64bit Production

Copyright (c) 2000, 2009, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
Connected.
DGMGRL> start observer;
Observer started

Once that the observer is running, let me enable FSFO. To do so, connect to the broker configuration and execute “enable fast_start failover” command.

 

DGMGRL> enable fast_start failover;
Enabled.
DGMGRL>
DGMGRL> show configuration;

Configuration - dgtest

  Protection Mode: MaxAvailability
  Databases:
    srprim - Primary database
    srpstb - (*) Physical standby database

Fast-Start Failover: ENABLED

Configuration Status:
SUCCESS

It’s clear from the above outcome that FSFO is enabled.
Let’s get the details of the properties set for FSFO.

 

DGMGRL> show fast_start failover;

Fast-Start Failover: ENABLED

  Threshold:        60 seconds
  Target:           srpstb
  Observer:         ora1-3.mydomain
  Lag Limit:        30 seconds (not in use)
  Shutdown Primary: TRUE
  Auto-reinstate:   TRUE

Configurable Failover Conditions
  Health Conditions:
    Corrupted Controlfile          YES
    Corrupted Dictionary           YES
    Inaccessible Logfile            NO
    Stuck Archiver                  NO
    Datafile Offline               YES

  Oracle Error Conditions:
    (none)

It can be noticed that FastStartFailoverThreshold is set to 60 seconds with FastStartFailoverTarget as SRPSTB, Observer being running at “ora1-3” host, FastStartFailoverLaglimit with the default value (30 seconds) which is currently not used in this configuration as the protection mode is set to MaxAvailability and most importantly FastStartFailoverPmyShutdown and FastStartFailoverAutoReinstate being set to TRUE.

 

FSFO can also be triggered with certain additional (optional) conditions. There has been no conditions specified from my end. So the values you see above are the default ones.

 

FSFO will occur if the Controlfile is corrupted or if the dictionary is corrupted or if a datafile is offline due to write error. In addition, you can configure other conditions such as Struck archiver (FSFO will occur if archive process is unable to archive the redo due to write error or the disk being full), Inaccessible logfile (LGWR is unable to write to the redo logs due to write error).

Query the primary database to check the FSFO and the observer status.

 

SQL> select  FS_FAILOVER_STATUS,FS_FAILOVER_OBSERVER_PRESENT   from v$database;

FS_FAILOVER_STATUS     FS_FAIL
---------------------- -------
SYNCHRONIZED           YES

To simulate a fast start failover, I crash the primary database by shutting it down using the “Abort” clause so that it looses the connectivity with the observer and the standby database.

 

SQL> shut abort
ORACLE instance shut down.

Now on checking the status of the configuration, Oracle throws out the error message that the primary database is unavailable and cannot determine it’s status.

 

DGMGRL> show configuration;

Configuration - dgtest

  Protection Mode: MaxAvailability
  Databases:
    srprim - Primary database
    srpstb - (*) Physical standby database

Fast-Start Failover: ENABLED

Configuration Status:
ORA-01034: ORACLE not available
ORA-16625: cannot reach database "srprim"
DGM-17017: unable to determine configuration status

The observer did not immediately trigger the failover as it has to wait for the FastStartFailoverThreshold number of seconds.
After a few seconds of wait, check the configuration status again. FSFO is in progress as seen below.

 

  
DGMGRL> show configuration;

Configuration - dgtest

  Protection Mode: MaxAvailability
  Databases:
    srpstb - Primary database
    srprim - (*) Physical standby database (disabled)

Fast-Start Failover: ENABLED

Configuration Status:
ORA-16610: command "FAILOVER TO srpstb" in progress
DGM-17017: unable to determine configuration status

It’s good to see below that fast start failover has occured and “srpstb” is the primary database.

 

DGMGRL> show configuration;

Configuration - dgtest

  Protection Mode: MaxAvailability
  Databases:
    srpstb - Primary database
      Warning: ORA-16817: unsynchronized fast-start failover configuration

    srprim - (*) Physical standby database (disabled)
      ORA-16661: the standby database needs to be reinstated

Fast-Start Failover: ENABLED

Configuration Status:
WARNING

Reinstating Former Primary database as New Standby database:

 

Now, that failover occurred, I’d like to reinstate the former primary database “srprim”. Since FastStartFailoverAutoReinstate was set to True, the observer will

reinstate the former primary database automatically once it is up.
But the below portion is a snippet of reinstating a former primary database as a new standby database if in case FastStartFailoverAutoReinstate was set to FALSE .
Connect to the former primary database “srprim” and mount it up.

 

[oracle@ora1-1 ~]$ sqlplus sys/oracle@srprim as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Thu Oct 15 19:30:46 2015

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup mount
ORACLE instance started.

Total System Global Area  943669248 bytes
Fixed Size                  2232128 bytes
Variable Size             641728704 bytes
Database Buffers          293601280 bytes
Redo Buffers                6107136 bytes
Database mounted.

Now connect to the broker configuration with the new primary database and execute “reinstate database ‘<former primary database>'”.
In my case it is : “reinstate databsae ‘srprim’;”

 

[oracle@ora1-2 ~]$ dgmgrl sys/oracle@srpstb
DGMGRL for Linux: Version 11.2.0.2.0 - 64bit Production

Copyright (c) 2000, 2009, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
Connected.
DGMGRL> reinstate database 'srprim';
Reinstating database "srprim", please wait...
Reinstatement of database "srprim" succeeded
DGMGRL>

Let’s check the configuration.

 

DGMGRL> show configuration;

Configuration - dgtest

  Protection Mode: MaxAvailability
  Databases:
    srpstb - Primary database
    srprim - (*) Physical standby database

Fast-Start Failover: ENABLED

Configuration Status:
SUCCESS

DGMGRL>

Good to see that srprim has been converted to the new standby database.

 

 

 

COPYRIGHT

© Shivananda Rao P, 2012, 2013, 2014, 2015. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Shivananda Rao and http://www.shivanandarao-oracle.com with appropriate and specific direction to the original content.

 

 

DISCLAIMER

The views expressed here are my own and do not necessarily reflect the views of any other individual, business entity, or organization. The views expressed by visitors on this blog are theirs solely and may not reflect mine.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: