--- schema_version: "1" id: 92c5505346eb616cc8384c4ae30a27ec1fea0efd384161812d78c9f7026f3254 title: Automatic App Restarts collection: mulesoft version: current locale: en-us portal_url: https://docs.salesforce.com/mulesoft/current/en-us/cloudhub-2/ch2-understand-app-restart.md canonical_url: https://docs.mulesoft.com/cloudhub-2/ch2-understand-app-restart source_site: docs.mulesoft.com content_type: documentation content_hash: 755cbd17afd1c6891716f8eeb24e9244aa1fe478c0347d51c574e02b859b2c83 description: MuleSoft Documentation Site latest: true --- --- product: CloudHub 2.0 --- # Automatic App Restarts > For the full documentation index, see: https://docs.mulesoft.com/llms.txt CloudHub 2.0 restarts your Mule applications during infrastructure failure or maintenance, security patching, replica crashes, or replica rebalancing. The CloudHub 2.0 infrastructure gracefully shuts down by draining connections, transitioning traffic to a new instance of your application replica before terminating the older instance. To ensure business resiliency, design your application to be idempotent and deploy across multiple replicas with [clustering](https://docs.mulesoft.com/cloudhub-2/ch2-clustering) enabled with a layer of persistence ([Object Store v2](https://docs.mulesoft.com/object-store/)). ## Understand Local Storage Limitations CloudHub 2.0 replica local storage is ephemeral. Unlike local development environments, CloudHub 2.0 clears local data when an app restarts. This limitation affects: Persistent object stores Configuring `persistent="true"` writes data to local disk. This works in Anypoint Studio and standalone Mule runtimes, but data doesn’t persist across CloudHub 2.0 restarts. Use [Object Store v2](https://docs.mulesoft.com/object-store/) for persistent storage. Local file operations The File connector and other file-based operations write to the replica’s local disk. CloudHub 2.0 deletes these files when the app restarts. Temporary files and caches App-generated files, custom logs, and local cache data don’t persist. To ensure data persistence in CloudHub 2.0: - Use [Object Store v2](https://docs.mulesoft.com/object-store/) instead of persistent object stores for app state and synchronization data. - Store files in external cloud storage services. - Design stateless integrations by persisting data in external databases or messaging queues. For more information about designing resilient applications, see [CloudHub 2.0 High Availability and Disaster Recovery](https://docs.mulesoft.com/cloudhub-2/ch2-ha-dr). ## Shutdown Timeout Behavior CloudHub 2.0 overrides the Mule runtime’s default `shutdownTimeout` value to avoid disruption for long-running flows. By default, the Mule runtime sets `shutdownTimeout` to five seconds. CloudHub 2.0 extends the shutdown grace period to accommodate applications that require more time to complete in-flight processing. For infrastructure-triggered restarts (such as infrastructure failure, maintenance, security patching, private space upgrades, or replica rebalancing): - CloudHub 2.0 provides a grace period up to a maximum of 12 hours. - If no custom `shutdownTimeout` is configured, CloudHub 2.0 applies a 12-hour maximum grace period to ensure long-running flows complete without disruption. For user-triggered restarts (such as stop, restart, update, redeploy, or terminate actions): - CloudHub 2.0 honors the configured `shutdownTimeout` value from your application configuration. - CloudHub 2.0 doesn’t provide the extended 12-hour grace period. You can optionally configure the `shutdownTimeout` attribute in your Mule application configuration. See [Global Configurations Reference](https://docs.mulesoft.com/mule-runtime/latest/global-settings-configuration#global-configurations-reference). ```xml ``` ## Considerations Review these details about how each replica strategy manages application optimization: Single Replica with Recreate Strategy The sole replica is terminated first, allowing it to gracefully shut down. A new replica launches as soon as the previous replica enters Stopping status. Incoming traffic experiences a brief downtime. If the application schedule triggers during this downtime, the trigger is missed. Single Replica with Rolling Restart A new replica launches before the old replica begins its graceful shutdown. Two replicas run concurrently until the old replica is fully terminated. Once in Stopping status, the old replica stops receiving new connections or triggering application schedules. Multiple Replicas with Recreate or Rolling Restart At least two thirds of the replicas remain available throughout the optimization process. Some replicas are gracefully shut down and restarted on new infrastructure. ## Infrastructure Failure and Maintenance If the compute infrastructure suffers a failure, CloudHub 2.0 moves your application to a healthy infrastructure and restarts the application replicas. Likewise, during infrastructure maintenance, the platform moves application replicas running on a compute infrastructure that becomes impaired or end-of-life onto a new infrastructure to ensure availability. ## Karpenter Node Downscaling in Private Spaces In Private Spaces, CloudHub 2.0 uses Karpenter to manage node lifecycle. When Karpenter downscales nodes to remove underused capacity, CloudHub 2.0 restarts applications running on those nodes as replicas move to remaining nodes. Restarts occur once per week at most. Each private space has a 4-hour consolidation window. CloudHub 2.0 randomizes the start time per space so it doesn’t restart all spaces in a region at once. The 4-hour block can start anytime between Friday 10:00 PM and Saturday 2:00 AM in the Private Space region’s local time zone, so restarts can occur anytime from Friday 10:00 PM through Saturday 6:00 AM. > [!NOTE] Apps in a Private Space can restart during Karpenter node downscaling in this weekly > window. These restarts are routine maintenance, not an outage. Design for resiliency so that > restarts have minimal impact. See > [CloudHub 2.0 High Availability and Disaster Recovery](https://docs.mulesoft.com/cloudhub-2/ch2-ha-dr). ## Security Patching To mitigate security vulnerabilities, CloudHub 2.0 patches and restarts applications every month. See the [Critical Security Vulnerabilities](https://www.salesforce.com/en-us/wp-content/uploads/sites/4/documents/legal/Agreements/versioning-back-support-policy.pdf) policy. ## Replica Crash If an application replica crashes for any reason, CloudHub 2.0 automatically restarts it. If it continues to fail to start, the platform attempts to restart it using an exponential backoff policy. ## Replica Rebalancing The CloudHub 2.0 infrastructure optimizes application efficiency through replica rebalancing. This process gracefully moves and restarts replicas, provided there are at least two replicas available. During rebalancing, a maximum of 30% of an application’s replicas are rebalanced concurrently. The system rounds up this percentage to the number of replicas that can be disrupted. For example, if your application has two replicas, a disruption affects one replica, exceeding the configured 30%. This table shows the potential impact of rebalancing operations on your applications based on the number of replicas.

Total Application Replicas	Maximum Replicas Disrupted
1	1
2	1
3	1
4	2
5	2
6	2
7	3
8	3
9	3
10	3
11	4
12	4
13	4
14	5
15	5
16	5

This method ensures that the application’s configuration, including the Mule runtime version and the infrastructure, remains consistent after all restarts, except for restarts occurring after a security patching. For more details about security patching, see [CloudHub 2.0 Operating System Patch Updates](https://docs.mulesoft.com/cloudhub-2/ch2-patch-updates).