# mulesoft Runtime Fabric - Full Documentation
---
product: Runtime Fabric
version: 2.6
---
# Allocating Resource for Application Deployment on Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying a Mule application to Anypoint Runtime Fabric, determine the correct number of
resources to allocate. Determining resource allocation is also important when configuring the
internal load balancers on Anypoint Runtime Fabrics.
When a Mule application is deployed to Runtime Fabric, the application is deployed with its own Mule
runtime engine (Mule). The number of replicas, or instances of that application and runtime, are
also specified. The resources available for each replica are determined by the values you set when
deploying an application.
The following performance information is based on clusters with controller nodes using custom Intel
Xeon E5-2676 v3 Haswell processors. The processors run at a base clock rate of 2.4 GHz and can go as
high as 3.0 GHz with Intel Turbo Boost. The load generator was hosted on a separate cluster in the
same region.
You can allocate the following resources when deploying an application:
- vCPU cores
- `Reserved vCPU`
The amount of vCPU guaranteed to the application and reserved for its use.
- `vCPU Limit`
The maximum amount of vCPU the application can use (the level to which it can burst). This is
shared CPU on the worker node.
- Memory
Review [MuleSoft licensing requirements](https://docs.mulesoft.com/runtime-fabric/2.6/limitations-self#license-compliance) for applications
deployed on Runtime Fabric instances.
## vCPU Allocation
When the `Reserved vCPU` and `vCPU Limit` are equal, the CPU on the worker node is allocated in a
guaranteed model. The container always has the specified amount of CPU available. The vCPU is
reserved for each replica of the application and is guaranteed regardless of what else is running on
Runtime Fabric.
Guaranteed CPU allocation allows applications to perform consistently by ensuring there is no CPU
contention between other applications. It guarantees performance in all cases.
When the value of `vCPU Limit` is set higher than the value of `Reserved vCPU`, the application can
burst up to the either the vCPU limit value or the amount of unreserved vCPU on the worker node,
whichever is less.
When estimating the CPU and memory required for worker nodes, Anypoint Monitoring agent requires at
least 0.05 vCPU cores and 50 Mi of memory for Anypoint Monitoring. This container is used to stream
application metrics to the control plane and cannot be disabled.
## CPU Bursting
When using CPU bursting, you must consider the following factors:
- Minimum reserved vCPU: 0.02 vCPU.
- Minimum vCPU limit: 0.07 vCPU.
Resource intensive applications might require additional CPU capacity. If an application
deployment fails using the minimum vCPU limit, increase the vCPU limit and redeploy the
application.
- The CPU limit is upper-bounded by the CPU cores provided on the worker nodes.
- The maximum recommended utilization per CPU core is 20 - 25 simple applications and API gateways.
More complex applications with higher usage may need more capacity.
- Runtime Fabric runs a small number of services on worker nodes. These services consume between 0.3
and 0.5 vCPU, depending on whether log forwarding is enabled for Anypoint Monitoring.
- Each application replica must reserve 50m CPU and 150Mi memory for the container that runs
Anypoint Monitoring. This container streams application metrics to the control plane and cannot be
disabled.
- Applications compete for the unallocated CPU remaining on their worker node. As more applications
are deployed on a worker node, less unallocated CPU is available to share with other applications
and can result in poor performance. The following strategies can be considered for minimizing CPU
contention with your applications:
- Deploy nightly batch applications with other applications that have peak load during the day.
- Deploy multiple replicas of an application so that they are distributed across worker nodes to
maximize the surface area for unallocated CPU.
Regardless of what approach is chosen, make sure you run tests to validate performance.
## Memory Allocation
The minimum values for the amount of memory allocated to each replica of a Mule application or API
gateway are the following:
- 0.7 GiB memory for Mule 4
- 0.5 GiB memory for Mule 3
When Anypoint Monitoring is enabled, 50MB memory is reserved for running the Anypoint Monitoring
sidecar containers, which can burst up to 70MB for each application deployed when processing metrics
data.
## Calculating Memory Allocation on CloudHub and Runtime Fabric
Anypoint Platform allocates native and heap memory for a deployed application. Heap memory is the
portion of the total memory that is made available to the Mule runtime and the application. Heap
memory is used for tasks like handling payload. Anypoint Platform allocates native memory for
deployed applications through the JVM, taking into account the standard practices of Java memory
management.
Both CloudHub and Anypoint Runtime Fabric allocate both types of memory. However, they differ in how
the memory allocation of each memory type is calculated.
- Runtime Fabric lists the total memory available for an application.
The available heap memory is approximately half of the total memory allocated to an application.
- CloudHub describes minimum memory requirements in terms of the heap memory available to an
application.
## Application Startup Times
The startup time for a Mule application is correlated to the total number of vCPU cores the
application has access to. The following lists approximate startup times for a simple Mule proxy
application performing processing on a payload:
| vCPU Cores | Approximate Startup Time |
|---|
| |
| |
| |
| |
## Application Performance
The resources allocated to your Mule application determine the application’s performance. The
following table lists approximate values for throughput based on the total number of vCPU cores
allocated for a single Mule application performing simple processing on a 10-KB payload:
| vCPU Cores | Concurrent Connections | Avg Response Time (ms) |
|---|
| | |
| | |
| | |
| | |
> [!NOTE] Run performance and load testing on your Mule applications to determine the number of
> resources to allocate.
## CPU Requirements for Keys and Certificates
Ensure that you allocate enough CPU resources to support a minmum of 10 PEM/P12 or 8 JKS/JCEKS
certificates. The number of recommended cores are:
## Key Types
RSA keys are the most common type of keys. RSA keys of 2K length offer the best compromise between
security and performance.
> [!NOTE] RSA keys larger than 2K protect against brute force cracking and are appropriate for
> certificates that have expirations of many years. However, whenever key length is doubled, for
> example, from 2k to 4k, performance is reduced by a factor greater than 6.
ECDSA keys are also supported. In most cases, ECDSA doubles the performance of a 2K RSA key.
Supported curves are:
- secp521r1 (P-521)
- secp384r1 (P-384)
- secp256r1 (also known as prime256v1 (P-256))
## See Also
- [Deploy a Mule Application to a Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.6/deploy-to-runtime-fabric)
- [View Runtime Fabric Health Details](https://docs.mulesoft.com/runtime-fabric/2.6/view-health)
---
product: Runtime Fabric
version: 2.7
---
# Allocating Resource for Application Deployment on Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying a Mule application to Anypoint Runtime Fabric, determine the correct number of
resources to allocate. Determining resource allocation is also important when configuring the
internal load balancers on Anypoint Runtime Fabrics.
When a Mule application is deployed to Runtime Fabric, the application is deployed with its own Mule
runtime engine (Mule). The number of replicas, or instances of that application and runtime, are
also specified. The resources available for each replica are determined by the values you set when
deploying an application.
The following performance information is based on clusters with controller nodes using custom Intel
Xeon E5-2676 v3 Haswell processors. The processors run at a base clock rate of 2.4 GHz and can go as
high as 3.0 GHz with Intel Turbo Boost. The load generator was hosted on a separate cluster in the
same region.
You can allocate the following resources when deploying an application:
- vCPU cores
- `Reserved vCPU`
The amount of vCPU guaranteed to the application and reserved for its use.
- `vCPU Limit`
The maximum amount of vCPU the application can use (the level to which it can burst). This is
shared CPU on the worker node.
- Memory
Review [MuleSoft licensing requirements](https://docs.mulesoft.com/runtime-fabric/2.7/limitations-self#license-compliance) for applications
deployed on Runtime Fabric instances.
## vCPU Allocation
When the `Reserved vCPU` and `vCPU Limit` are equal, the CPU on the worker node is allocated in a
guaranteed model. The container always has the specified amount of CPU available. The vCPU is
reserved for each replica of the application and is guaranteed regardless of what else is running on
Runtime Fabric.
Guaranteed CPU allocation allows applications to perform consistently by ensuring there is no CPU
contention between other applications. It guarantees performance in all cases.
When the value of `vCPU Limit` is set higher than the value of `Reserved vCPU`, the application can
burst up to the either the vCPU limit value or the amount of unreserved vCPU on the worker node,
whichever is less.
When estimating the CPU and memory required for worker nodes, Anypoint Monitoring agent requires at
least 0.05 vCPU cores and 50 Mi of memory for Anypoint Monitoring. This container is used to stream
application metrics to the control plane and cannot be disabled.
## CPU Bursting
When using CPU bursting, you must consider the following factors:
- Minimum reserved vCPU: 0.02 vCPU.
- Minimum vCPU limit: 0.07 vCPU.
Resource intensive applications might require additional CPU capacity. If an application
deployment fails using the minimum vCPU limit, increase the vCPU limit and redeploy the
application.
- The CPU limit is upper-bounded by the CPU cores provided on the worker nodes.
- The maximum recommended utilization per CPU core is 20 - 25 simple applications and API gateways.
More complex applications with higher usage may need more capacity.
- Runtime Fabric runs a small number of services on worker nodes. These services consume between 0.3
and 0.5 vCPU, depending on whether log forwarding is enabled for Anypoint Monitoring.
- Each application replica must reserve 50m CPU and 150Mi memory for the container that runs
Anypoint Monitoring. This container streams application metrics to the control plane and cannot be
disabled.
- Applications compete for the unallocated CPU remaining on their worker node. As more applications
are deployed on a worker node, less unallocated CPU is available to share with other applications
and can result in poor performance. The following strategies can be considered for minimizing CPU
contention with your applications:
- Deploy nightly batch applications with other applications that have peak load during the day.
- Deploy multiple replicas of an application so that they are distributed across worker nodes to
maximize the surface area for unallocated CPU.
Regardless of what approach is chosen, make sure you run tests to validate performance.
## Memory Allocation
The minimum values for the amount of memory allocated to each replica of a Mule application or API
gateway are the following:
- 0.7 GiB memory for Mule 4
- 0.5 GiB memory for Mule 3
When Anypoint Monitoring is enabled, 50MB memory is reserved for running the Anypoint Monitoring
sidecar containers, which can burst up to 70MB for each application deployed when processing metrics
data.
## Calculating Memory Allocation on CloudHub and Runtime Fabric
Anypoint Platform allocates native and heap memory for a deployed application. Heap memory is the
portion of the total memory that is made available to the Mule runtime and the application. Heap
memory is used for tasks like handling payload. Anypoint Platform allocates native memory for
deployed applications through the JVM, taking into account the standard practices of Java memory
management.
Both CloudHub and Anypoint Runtime Fabric allocate both types of memory. However, they differ in how
the memory allocation of each memory type is calculated.
- Runtime Fabric lists the total memory available for an application.
The available heap memory is approximately half of the total memory allocated to an application.
- CloudHub describes minimum memory requirements in terms of the heap memory available to an
application.
## Application Startup Times
The startup time for a Mule application is correlated to the total number of vCPU cores the
application has access to. The following lists approximate startup times for a simple Mule proxy
application performing processing on a payload:
| vCPU Cores | Approximate Startup Time |
|---|
| |
| |
| |
| |
## Application Performance
The resources allocated to your Mule application determine the application’s performance. The
following table lists approximate values for throughput based on the total number of vCPU cores
allocated for a single Mule application performing simple processing on a 10-KB payload:
| vCPU Cores | Concurrent Connections | Avg Response Time (ms) |
|---|
| | |
| | |
| | |
| | |
> [!NOTE] Run performance and load testing on your Mule applications to determine the number of
> resources to allocate.
## CPU Requirements for Keys and Certificates
Ensure that you allocate enough CPU resources to support a minmum of 10 PEM/P12 or 8 JKS/JCEKS
certificates. The number of recommended cores are:
## Key Types
RSA keys are the most common type of keys. RSA keys of 2K length offer the best compromise between
security and performance.
> [!NOTE] RSA keys larger than 2K protect against brute force cracking and are appropriate for
> certificates that have expirations of many years. However, whenever key length is doubled, for
> example, from 2k to 4k, performance is reduced by a factor greater than 6.
ECDSA keys are also supported. In most cases, ECDSA doubles the performance of a 2K RSA key.
Supported curves are:
- secp521r1 (P-521)
- secp384r1 (P-384)
- secp256r1 (also known as prime256v1 (P-256))
## See Also
- [Deploy a Mule Application to a Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.7/deploy-to-runtime-fabric)
- [View Runtime Fabric Health Details](https://docs.mulesoft.com/runtime-fabric/2.7/view-health)
---
product: Runtime Fabric
version: 2.10
---
# Allocating Resources for Application Deployment on Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying a Mule application to Anypoint Runtime Fabric, determine the correct number of
resources to allocate. Determining resource allocation is also important when configuring the
internal load balancers on Anypoint Runtime Fabrics.
When a Mule application is deployed to Runtime Fabric, the application is deployed with its own Mule
runtime engine (Mule). The number of replicas, or instances of that application and runtime, are
also specified. The resources available for each replica are determined by the values you set when
deploying an application.
The following performance information is based on clusters with controller nodes using custom Intel
Xeon E5-2676 v3 Haswell processors. The processors run at a base clock rate of 2.4 GHz and can go as
high as 3.0 GHz with Intel Turbo Boost. The load generator was hosted on a separate cluster in the
same region.
You can allocate the following resources when deploying an application:
- vCPU cores
- `Reserved vCPU`
The amount of vCPU guaranteed to the application and reserved for its use.
- `vCPU Limit`
The maximum amount of vCPU the application can use (the level to which it can burst). This is
shared CPU on the worker node.
- Memory
Review [MuleSoft licensing requirements](https://docs.mulesoft.com/runtime-fabric/2.10/limitations-self#license-compliance) for applications
deployed on Runtime Fabric instances.
## vCPU Allocation
When the `Reserved vCPU` and `vCPU Limit` are equal, the CPU on the worker node is allocated in a
guaranteed model. The container always has the specified amount of CPU available. The vCPU is
reserved for each replica of the application and is guaranteed regardless of what else is running on
Runtime Fabric.
Guaranteed CPU allocation allows applications to perform consistently by ensuring there is no CPU
contention between other applications. It guarantees performance in all cases.
When the value of `vCPU Limit` is set higher than the value of `Reserved vCPU`, the application can
burst up to the either the vCPU limit value or the amount of unreserved vCPU on the worker node,
whichever is less.
When estimating the CPU and memory required for worker nodes, Anypoint Monitoring agent requires at
least 0.05 vCPU cores and 50 Mi of memory for Anypoint Monitoring. This container is used to stream
application metrics to the control plane and cannot be disabled.
## CPU Bursting
When using CPU bursting, you must consider the following factors:
- Minimum reserved vCPU: 0.02 vCPU.
- Minimum vCPU limit: 0.07 vCPU.
Resource intensive applications might require additional CPU capacity. If an application
deployment fails using the minimum vCPU limit, increase the vCPU limit and redeploy the
application.
- The CPU limit is upper-bounded by the CPU cores provided on the worker nodes.
- The maximum recommended utilization per CPU core is 20 - 25 simple applications and API gateways.
More complex applications with higher usage may need more capacity.
- Runtime Fabric runs a small number of services on worker nodes. These services consume between 0.3
and 0.5 vCPU, depending on whether log forwarding is enabled for Anypoint Monitoring.
- Each application replica must reserve 50m CPU and 150Mi memory for the container that runs
Anypoint Monitoring. This container streams application metrics to the control plane and cannot be
disabled.
- Applications compete for the unallocated CPU remaining on their worker node. As more applications
are deployed on a worker node, less unallocated CPU is available to share with other applications
and can result in poor performance. The following strategies can be considered for minimizing CPU
contention with your applications:
- Deploy nightly batch applications with other applications that have peak load during the day.
- Deploy multiple replicas of an application so that they are distributed across worker nodes to
maximize the surface area for unallocated CPU.
Regardless of what approach is chosen, make sure you run tests to validate performance.
## Memory Allocation
The minimum values for the amount of memory allocated to each replica of a Mule application or API
gateway is:
- 0.7 GiB memory for Mule 4
When Anypoint Monitoring is enabled, 50MB memory is reserved for running the Anypoint Monitoring
sidecar containers, which can burst up to 70MB for each application deployed when processing metrics
data.
## Calculating Memory Allocation on CloudHub and Runtime Fabric
Anypoint Platform allocates native and heap memory for a deployed application. Heap memory is the
portion of the total memory that is made available to the Mule runtime and the application. Heap
memory is used for tasks like handling payload. Anypoint Platform allocates native memory for
deployed applications through the JVM, taking into account the standard practices of Java memory
management.
Both CloudHub and Anypoint Runtime Fabric allocate both types of memory. However, they differ in how
the memory allocation of each memory type is calculated.
- Runtime Fabric lists the total memory available for an application.
The available heap memory is approximately half of the total memory allocated to an application.
- CloudHub describes minimum memory requirements in terms of the heap memory available to an
application.
## Application Startup Times
The startup time for a Mule application is correlated to the total number of vCPU cores the
application has access to. The following lists approximate startup times for a simple Mule proxy
application performing processing on a payload:
| vCPU Cores | Approximate Startup Time |
|---|
| |
| |
| |
| |
## Application Performance
The resources allocated to your Mule application determine the application’s performance. The
following table lists approximate values for throughput based on the total number of vCPU cores
allocated for a single Mule application performing simple processing on a 10-KB payload:
| vCPU Cores | Concurrent Connections | Avg Response Time (ms) |
|---|
| | |
| | |
| | |
| | |
> [!NOTE] Run performance and load testing on your Mule applications to determine the number of
> resources to allocate.
## CPU Requirements for Keys and Certificates
Ensure that you allocate enough CPU resources to support a minmum of 10 PEM/P12 or 8 JKS/JCEKS
certificates. The number of recommended cores are:
## Key Types
RSA keys are the most common type of keys. RSA keys of 2K length offer the best compromise between
security and performance.
> [!NOTE] RSA keys larger than 2K protect against brute force cracking and are appropriate for
> certificates that have expirations of many years. However, whenever key length is doubled, for
> example, from 2k to 4k, performance is reduced by a factor greater than 6.
ECDSA keys are also supported. In most cases, ECDSA doubles the performance of a 2K RSA key.
Supported curves are:
- secp521r1 (P-521)
- secp384r1 (P-384)
- secp256r1 (also known as prime256v1 (P-256))
## See Also
- [Deploy a Mule Application to a Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.10/deploy-to-runtime-fabric)
- [View Runtime Fabric Health Details](https://docs.mulesoft.com/runtime-fabric/2.10/view-health)
---
product: Runtime Fabric
version: 2.11
---
# Allocating Resources for Application Deployment on Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying a Mule application to Anypoint Runtime Fabric, determine the correct number of
resources to allocate. Determining resource allocation is also important when configuring the
internal load balancers on Anypoint Runtime Fabrics.
When a Mule application is deployed to Runtime Fabric, the application is deployed with its own Mule
runtime engine (Mule). The number of replicas, or instances of that application and runtime, are
also specified. The resources available for each replica are determined by the values you set when
deploying an application.
The following performance information is based on clusters with controller nodes using custom Intel
Xeon E5-2676 v3 Haswell processors. The processors run at a base clock rate of 2.4 GHz and can go as
high as 3.0 GHz with Intel Turbo Boost. The load generator was hosted on a separate cluster in the
same region.
You can allocate these resources when deploying an application:
- vCPU cores
- `Reserved vCPU`
The amount of vCPU guaranteed to the application and reserved for its use.
- `vCPU Limit`
The maximum amount of vCPU the application can use (the level to which it can burst). This is
shared CPU on the worker node.
- Memory
Review [MuleSoft licensing requirements](https://docs.mulesoft.com/runtime-fabric/2.11/limitations-self#license-compliance) for applications
deployed on Runtime Fabric instances.
## vCPU Allocation
When the `Reserved vCPU` and `vCPU Limit` are equal, the CPU on the worker node is allocated in a
guaranteed model. The container always has the specified amount of CPU available. The vCPU is
reserved for each replica of the application and is guaranteed regardless of what else is running on
Runtime Fabric.
Guaranteed CPU allocation allows applications to perform consistently by ensuring there is no CPU
contention between other applications. It guarantees performance in all cases.
When the value of `vCPU Limit` is set higher than the value of `Reserved vCPU`, the application can
burst up to the either the vCPU limit value or the amount of unreserved vCPU on the worker node,
whichever is less.
When estimating the CPU and memory required for worker nodes, Anypoint Monitoring agent requires at
least 0.05 vCPU cores and 50 Mi of memory for Anypoint Monitoring. This container is used to stream
application metrics to the control plane and cannot be disabled.
## CPU Bursting
When using CPU bursting, you must consider these factors:
- Minimum reserved vCPU: 0.02 vCPU.
- Minimum vCPU limit: 0.07 vCPU.
Resource intensive applications might require additional CPU capacity. If an application
deployment fails using the minimum vCPU limit, increase the vCPU limit and redeploy the
application.
- The CPU limit is upper-bounded by the CPU cores provided on the worker nodes.
- The maximum recommended utilization per CPU core is 20 - 25 simple applications and API gateways.
More complex applications with higher usage may need more capacity.
- Runtime Fabric runs a small number of services on worker nodes. These services consume between 0.3
and 0.5 vCPU, depending on whether log forwarding is enabled for Anypoint Monitoring.
- Each application replica must reserve 50m CPU and 150Mi memory for the container that runs
Anypoint Monitoring. This container streams application metrics to the control plane and cannot be
disabled.
- Applications compete for the unallocated CPU remaining on their worker node. As more applications
are deployed on a worker node, less unallocated CPU is available to share with other applications
and can result in poor performance. The following strategies can be considered for minimizing CPU
contention with your applications:
- Deploy nightly batch applications with other applications that have peak load during the day.
- Deploy multiple replicas of an application so that they are distributed across worker nodes to
maximize the surface area for unallocated CPU.
Regardless of what approach is chosen, make sure you run tests to validate performance.
## Memory Allocation
The minimum values for the amount of memory allocated to each replica of a Mule application or API
gateway is:
- 0.7 GiB memory for Mule 4
When Anypoint Monitoring is enabled, 50MB memory is reserved for running the Anypoint Monitoring
sidecar containers, which can burst up to 70MB for each application deployed when processing metrics
data.
## Calculating Memory Allocation on CloudHub and Runtime Fabric
Anypoint Platform allocates native and heap memory for a deployed application. Heap memory is the
portion of the total memory that is made available to the Mule runtime and the application. Heap
memory is used for tasks like handling payload. Anypoint Platform allocates native memory for
deployed applications through the JVM, taking into account the standard practices of Java memory
management.
Both CloudHub and Anypoint Runtime Fabric allocate both types of memory. However, they differ in how
the memory allocation of each memory type is calculated.
- Runtime Fabric lists the total memory available for an application.
The available heap memory is approximately half of the total memory allocated to an application.
- CloudHub describes minimum memory requirements in terms of the heap memory available to an
application.
## Application Startup Times
The startup time for a Mule application is correlated to the total number of vCPU cores the
application has access to. The following lists approximate startup times for a simple Mule proxy
application performing processing on a payload:
| vCPU Cores | Approximate Startup Time |
|---|
| |
| |
| |
| |
## Application Performance
The resources allocated to your Mule application determine the application’s performance. The
following table lists approximate values for throughput based on the total number of vCPU cores
allocated for a single Mule application performing simple processing on a 10-KB payload:
| vCPU Cores | Concurrent Connections | Avg Response Time (ms) |
|---|
| | |
| | |
| | |
| | |
> [!NOTE] Run performance and load testing on your Mule applications to determine the number of
> resources to allocate.
## CPU Requirements for Keys and Certificates
Ensure that you allocate enough CPU resources to support a minmum of 10 PEM/P12 or 8 JKS/JCEKS
certificates. The number of recommended cores are:
## Key Types
RSA keys are the most common type of keys. RSA keys of 2K length offer the best compromise between
security and performance.
> [!NOTE] RSA keys larger than 2K protect against brute force cracking and are appropriate for
> certificates that have expirations of many years. However, whenever key length is doubled, for
> example, from 2k to 4k, performance is reduced by a factor greater than 6.
ECDSA keys are also supported. In most cases, ECDSA doubles the performance of a 2K RSA key.
Supported curves are:
- secp521r1 (P-521)
- secp384r1 (P-384)
- secp256r1 (also known as prime256v1 (P-256))
## See Also
- [Deploy a Mule Application to a Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.11/deploy-to-runtime-fabric)
- [View Runtime Fabric Health Details](https://docs.mulesoft.com/runtime-fabric/2.11/view-health)
---
product: Runtime Fabric
version: 2.8
---
# Allocating Resources for Application Deployment on Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying a Mule application to Anypoint Runtime Fabric, determine the correct number of
resources to allocate. Determining resource allocation is also important when configuring the
internal load balancers on Anypoint Runtime Fabrics.
When a Mule application is deployed to Runtime Fabric, the application is deployed with its own Mule
runtime engine (Mule). The number of replicas, or instances of that application and runtime, are
also specified. The resources available for each replica are determined by the values you set when
deploying an application.
The following performance information is based on clusters with controller nodes using custom Intel
Xeon E5-2676 v3 Haswell processors. The processors run at a base clock rate of 2.4 GHz and can go as
high as 3.0 GHz with Intel Turbo Boost. The load generator was hosted on a separate cluster in the
same region.
You can allocate the following resources when deploying an application:
- vCPU cores
- `Reserved vCPU`
The amount of vCPU guaranteed to the application and reserved for its use.
- `vCPU Limit`
The maximum amount of vCPU the application can use (the level to which it can burst). This is
shared CPU on the worker node.
- Memory
Review [MuleSoft licensing requirements](https://docs.mulesoft.com/runtime-fabric/2.8/limitations-self#license-compliance) for applications
deployed on Runtime Fabric instances.
## vCPU Allocation
When the `Reserved vCPU` and `vCPU Limit` are equal, the CPU on the worker node is allocated in a
guaranteed model. The container always has the specified amount of CPU available. The vCPU is
reserved for each replica of the application and is guaranteed regardless of what else is running on
Runtime Fabric.
Guaranteed CPU allocation allows applications to perform consistently by ensuring there is no CPU
contention between other applications. It guarantees performance in all cases.
When the value of `vCPU Limit` is set higher than the value of `Reserved vCPU`, the application can
burst up to the either the vCPU limit value or the amount of unreserved vCPU on the worker node,
whichever is less.
When estimating the CPU and memory required for worker nodes, Anypoint Monitoring agent requires at
least 0.05 vCPU cores and 50 Mi of memory for Anypoint Monitoring. This container is used to stream
application metrics to the control plane and cannot be disabled.
## CPU Bursting
When using CPU bursting, you must consider the following factors:
- Minimum reserved vCPU: 0.02 vCPU.
- Minimum vCPU limit: 0.07 vCPU.
Resource intensive applications might require additional CPU capacity. If an application
deployment fails using the minimum vCPU limit, increase the vCPU limit and redeploy the
application.
- The CPU limit is upper-bounded by the CPU cores provided on the worker nodes.
- The maximum recommended utilization per CPU core is 20 - 25 simple applications and API gateways.
More complex applications with higher usage may need more capacity.
- Runtime Fabric runs a small number of services on worker nodes. These services consume between 0.3
and 0.5 vCPU, depending on whether log forwarding is enabled for Anypoint Monitoring.
- Each application replica must reserve 50m CPU and 150Mi memory for the container that runs
Anypoint Monitoring. This container streams application metrics to the control plane and cannot be
disabled.
- Applications compete for the unallocated CPU remaining on their worker node. As more applications
are deployed on a worker node, less unallocated CPU is available to share with other applications
and can result in poor performance. The following strategies can be considered for minimizing CPU
contention with your applications:
- Deploy nightly batch applications with other applications that have peak load during the day.
- Deploy multiple replicas of an application so that they are distributed across worker nodes to
maximize the surface area for unallocated CPU.
Regardless of what approach is chosen, make sure you run tests to validate performance.
## Memory Allocation
The minimum values for the amount of memory allocated to each replica of a Mule application or API
gateway are the following:
- 0.7 GiB memory for Mule 4
- 0.5 GiB memory for Mule 3
When Anypoint Monitoring is enabled, 50MB memory is reserved for running the Anypoint Monitoring
sidecar containers, which can burst up to 70MB for each application deployed when processing metrics
data.
## Calculating Memory Allocation on CloudHub and Runtime Fabric
Anypoint Platform allocates native and heap memory for a deployed application. Heap memory is the
portion of the total memory that is made available to the Mule runtime and the application. Heap
memory is used for tasks like handling payload. Anypoint Platform allocates native memory for
deployed applications through the JVM, taking into account the standard practices of Java memory
management.
Both CloudHub and Anypoint Runtime Fabric allocate both types of memory. However, they differ in how
the memory allocation of each memory type is calculated.
- Runtime Fabric lists the total memory available for an application.
The available heap memory is approximately half of the total memory allocated to an application.
- CloudHub describes minimum memory requirements in terms of the heap memory available to an
application.
## Application Startup Times
The startup time for a Mule application is correlated to the total number of vCPU cores the
application has access to. The following lists approximate startup times for a simple Mule proxy
application performing processing on a payload:
| vCPU Cores | Approximate Startup Time |
|---|
| |
| |
| |
| |
## Application Performance
The resources allocated to your Mule application determine the application’s performance. The
following table lists approximate values for throughput based on the total number of vCPU cores
allocated for a single Mule application performing simple processing on a 10-KB payload:
| vCPU Cores | Concurrent Connections | Avg Response Time (ms) |
|---|
| | |
| | |
| | |
| | |
> [!NOTE] Run performance and load testing on your Mule applications to determine the number of
> resources to allocate.
## CPU Requirements for Keys and Certificates
Ensure that you allocate enough CPU resources to support a minimum of 10 PEM/P12 or 8 JKS/JCEKS
certificates. The number of recommended cores are:
## Key Types
RSA keys are the most common type of keys. RSA keys of 2K length offer the best compromise between
security and performance.
> [!NOTE] RSA keys larger than 2K protect against brute force cracking and are appropriate for
> certificates that have expirations of many years. However, whenever key length is doubled, for
> example, from 2k to 4k, performance is reduced by a factor greater than 6.
ECDSA keys are also supported. In most cases, ECDSA doubles the performance of a 2K RSA key.
Supported curves are:
- secp521r1 (P-521)
- secp384r1 (P-384)
- secp256r1 (also known as prime256v1 (P-256))
## See Also
- [Deploy a Mule Application to a Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.8/deploy-to-runtime-fabric)
- [View Runtime Fabric Health Details](https://docs.mulesoft.com/runtime-fabric/2.8/view-health)
---
product: Runtime Fabric
version: 2.9
---
# Allocating Resources for Application Deployment on Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying a Mule application to Anypoint Runtime Fabric, determine the correct number of
resources to allocate. Determining resource allocation is also important when configuring the
internal load balancers on Anypoint Runtime Fabrics.
When a Mule application is deployed to Runtime Fabric, the application is deployed with its own Mule
runtime engine (Mule). The number of replicas, or instances of that application and runtime, are
also specified. The resources available for each replica are determined by the values you set when
deploying an application.
The following performance information is based on clusters with controller nodes using custom Intel
Xeon E5-2676 v3 Haswell processors. The processors run at a base clock rate of 2.4 GHz and can go as
high as 3.0 GHz with Intel Turbo Boost. The load generator was hosted on a separate cluster in the
same region.
You can allocate the following resources when deploying an application:
- vCPU cores
- `Reserved vCPU`
The amount of vCPU guaranteed to the application and reserved for its use.
- `vCPU Limit`
The maximum amount of vCPU the application can use (the level to which it can burst). This is
shared CPU on the worker node.
- Memory
Review [MuleSoft licensing requirements](https://docs.mulesoft.com/runtime-fabric/2.9/limitations-self#license-compliance) for applications
deployed on Runtime Fabric instances.
## vCPU Allocation
When the `Reserved vCPU` and `vCPU Limit` are equal, the CPU on the worker node is allocated in a
guaranteed model. The container always has the specified amount of CPU available. The vCPU is
reserved for each replica of the application and is guaranteed regardless of what else is running on
Runtime Fabric.
Guaranteed CPU allocation allows applications to perform consistently by ensuring there is no CPU
contention between other applications. It guarantees performance in all cases.
When the value of `vCPU Limit` is set higher than the value of `Reserved vCPU`, the application can
burst up to the either the vCPU limit value or the amount of unreserved vCPU on the worker node,
whichever is less.
When estimating the CPU and memory required for worker nodes, Anypoint Monitoring agent requires at
least 0.05 vCPU cores and 50 Mi of memory for Anypoint Monitoring. This container is used to stream
application metrics to the control plane and cannot be disabled.
## CPU Bursting
When using CPU bursting, you must consider the following factors:
- Minimum reserved vCPU: 0.02 vCPU.
- Minimum vCPU limit: 0.07 vCPU.
Resource intensive applications might require additional CPU capacity. If an application
deployment fails using the minimum vCPU limit, increase the vCPU limit and redeploy the
application.
- The CPU limit is upper-bounded by the CPU cores provided on the worker nodes.
- The maximum recommended utilization per CPU core is 20 - 25 simple applications and API gateways.
More complex applications with higher usage may need more capacity.
- Runtime Fabric runs a small number of services on worker nodes. These services consume between 0.3
and 0.5 vCPU, depending on whether log forwarding is enabled for Anypoint Monitoring.
- Each application replica must reserve 50m CPU and 150Mi memory for the container that runs
Anypoint Monitoring. This container streams application metrics to the control plane and cannot be
disabled.
- Applications compete for the unallocated CPU remaining on their worker node. As more applications
are deployed on a worker node, less unallocated CPU is available to share with other applications
and can result in poor performance. The following strategies can be considered for minimizing CPU
contention with your applications:
- Deploy nightly batch applications with other applications that have peak load during the day.
- Deploy multiple replicas of an application so that they are distributed across worker nodes to
maximize the surface area for unallocated CPU.
Regardless of what approach is chosen, make sure you run tests to validate performance.
## Memory Allocation
The minimum values for the amount of memory allocated to each replica of a Mule application or API
gateway is:
- 0.7 GiB memory for Mule 4
When Anypoint Monitoring is enabled, 50MB memory is reserved for running the Anypoint Monitoring
sidecar containers, which can burst up to 70MB for each application deployed when processing metrics
data.
## Calculating Memory Allocation on CloudHub and Runtime Fabric
Anypoint Platform allocates native and heap memory for a deployed application. Heap memory is the
portion of the total memory that is made available to the Mule runtime and the application. Heap
memory is used for tasks like handling payload. Anypoint Platform allocates native memory for
deployed applications through the JVM, taking into account the standard practices of Java memory
management.
Both CloudHub and Anypoint Runtime Fabric allocate both types of memory. However, they differ in how
the memory allocation of each memory type is calculated.
- Runtime Fabric lists the total memory available for an application.
The available heap memory is approximately half of the total memory allocated to an application.
- CloudHub describes minimum memory requirements in terms of the heap memory available to an
application.
## Application Startup Times
The startup time for a Mule application is correlated to the total number of vCPU cores the
application has access to. The following lists approximate startup times for a simple Mule proxy
application performing processing on a payload:
| vCPU Cores | Approximate Startup Time |
|---|
| |
| |
| |
| |
## Application Performance
The resources allocated to your Mule application determine the application’s performance. The
following table lists approximate values for throughput based on the total number of vCPU cores
allocated for a single Mule application performing simple processing on a 10-KB payload:
| vCPU Cores | Concurrent Connections | Avg Response Time (ms) |
|---|
| | |
| | |
| | |
| | |
> [!NOTE] Run performance and load testing on your Mule applications to determine the number of
> resources to allocate.
## CPU Requirements for Keys and Certificates
Ensure that you allocate enough CPU resources to support a minmum of 10 PEM/P12 or 8 JKS/JCEKS
certificates. The number of recommended cores are:
## Key Types
RSA keys are the most common type of keys. RSA keys of 2K length offer the best compromise between
security and performance.
> [!NOTE] RSA keys larger than 2K protect against brute force cracking and are appropriate for
> certificates that have expirations of many years. However, whenever key length is doubled, for
> example, from 2k to 4k, performance is reduced by a factor greater than 6.
ECDSA keys are also supported. In most cases, ECDSA doubles the performance of a 2K RSA key.
Supported curves are:
- secp521r1 (P-521)
- secp384r1 (P-384)
- secp256r1 (also known as prime256v1 (P-256))
## See Also
- [Deploy a Mule Application to a Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.9/deploy-to-runtime-fabric)
- [View Runtime Fabric Health Details](https://docs.mulesoft.com/runtime-fabric/2.9/view-health)
---
product: Runtime Fabric
version: 3.0
is-latest-version: true
---
# Allocating Resources for Application Deployment on Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying a Mule application to Anypoint Runtime Fabric, determine the correct number of
resources to allocate. Determining resource allocation is also important when configuring the
internal load balancers on Anypoint Runtime Fabrics.
When a Mule application is deployed to Runtime Fabric, the application is deployed with its own Mule
runtime engine (Mule). The number of replicas, or instances of that application and runtime, are
also specified. The resources available for each replica are determined by the values you set when
deploying an application.
The following performance information is based on clusters with controller nodes using custom Intel
Xeon E5-2676 v3 Haswell processors. The processors run at a base clock rate of 2.4 GHz and can go as
high as 3.0 GHz with Intel Turbo Boost. The load generator was hosted on a separate cluster in the
same region.
You can allocate the following resources when deploying an application:
- vCPU cores
- `Reserved vCPU`
The amount of vCPU guaranteed to the application and reserved for its use.
- `vCPU Limit`
The maximum amount of vCPU the application can use (the level to which it can burst). This is
shared CPU on the worker node.
- Memory
Review [MuleSoft licensing requirements](https://docs.mulesoft.com/runtime-fabric/latest/limitations-self#license-compliance) for applications
deployed on Runtime Fabric instances.
## vCPU Allocation
When the `Reserved vCPU` and `vCPU Limit` are equal, the CPU on the worker node is allocated in a
guaranteed model. The container always has the specified amount of CPU available. The vCPU is
reserved for each replica of the application and is guaranteed regardless of what else is running on
Runtime Fabric.
Guaranteed CPU allocation allows applications to perform consistently by ensuring there is no CPU
contention between other applications. It guarantees performance in all cases.
When the value of `vCPU Limit` is set higher than the value of `Reserved vCPU`, the application can
burst up to the either the vCPU limit value or the amount of unreserved vCPU on the worker node,
whichever is less.
When estimating the CPU and memory required for worker nodes, each application replica includes an
Anypoint Monitoring sidecar container that streams application metrics to the control plane. The
resource allocation for this container varies depending on enabled features such as Anypoint
Monitoring logging and tracing. As a baseline, plan for at least 0.05 vCPU cores and 50 Mi of memory
per replica, though actual usage can be higher when additional monitoring features are enabled.
## CPU Bursting
When using CPU bursting, you must consider the following factors:
- Minimum reserved vCPU: 0.02 vCPU.
- Minimum vCPU limit: 0.07 vCPU.
Resource intensive applications might require additional CPU capacity. If an application
deployment fails using the minimum vCPU limit, increase the vCPU limit and redeploy the
application.
- The CPU limit is upper-bounded by the CPU cores provided on the worker nodes.
- The maximum recommended utilization per CPU core is 20 - 25 simple applications and API gateways.
More complex applications with higher usage may need more capacity.
- Runtime Fabric runs a small number of services on worker nodes. These services consume between 0.3
and 0.5 vCPU, depending on whether log forwarding is enabled for Anypoint Monitoring.
- Each application replica includes an Anypoint Monitoring sidecar container that streams
application metrics to the control plane and can’t be disabled. The CPU and memory reserved for
this container vary depending on the monitoring features enabled for the application (such as
tracing). As a baseline, plan for at least 50m CPU and 50Mi memory per replica, though actual
values can be higher.
- Applications compete for the unallocated CPU remaining on their worker node. As more applications
are deployed on a worker node, less unallocated CPU is available to share with other applications
and can result in poor performance. The following strategies can be considered for minimizing CPU
contention with your applications:
- Deploy nightly batch applications with other applications that have peak load during the day.
- Deploy multiple replicas of an application so that they are distributed across worker nodes to
maximize the surface area for unallocated CPU.
Regardless of what approach is chosen, make sure you run tests to validate performance.
## Memory Allocation
The minimum values for the amount of memory allocated to each replica of a Mule application or API
gateway is:
- 0.7 GiB memory for Mule 4
When Anypoint Monitoring is enabled, memory is reserved for the Anypoint Monitoring sidecar
container on each application replica. The actual memory allocation varies depending on enabled
monitoring features such as tracing. Plan for a baseline of at least 50Mi per replica, with higher
allocations when additional features are active.
## Calculating Memory Allocation on CloudHub and Runtime Fabric
Anypoint Platform allocates native and heap memory for a deployed application. Heap memory is the
portion of the total memory that is made available to the Mule runtime and the application. Heap
memory is used for tasks like handling payload. Anypoint Platform allocates native memory for
deployed applications through the JVM, taking into account the standard practices of Java memory
management.
Both CloudHub and Anypoint Runtime Fabric allocate both types of memory. However, they differ in how
the memory allocation of each memory type is calculated.
- Runtime Fabric lists the total memory available for an application.
The available heap memory is approximately half of the total memory allocated to an application.
- CloudHub describes minimum memory requirements in terms of the heap memory available to an
application.
## Application Startup Times
The startup time for a Mule application is correlated to the total number of vCPU cores the
application has access to. The following lists approximate startup times for a simple Mule proxy
application performing processing on a payload:
| vCPU Cores | Approximate Startup Time |
|---|
| |
| |
| |
| |
## Application Performance
The resources allocated to your Mule application determine the application’s performance. The
following table lists approximate values for throughput based on the total number of vCPU cores
allocated for a single Mule application performing simple processing on a 10-KB payload:
| vCPU Cores | Concurrent Connections | Avg Response Time (ms) |
|---|
| | |
| | |
| | |
| | |
> [!NOTE] Run performance and load testing on your Mule applications to determine the number of
> resources to allocate.
## CPU Requirements for Keys and Certificates
Ensure that you allocate enough CPU resources to support a minmum of 10 PEM/P12 or 8 JKS/JCEKS
certificates. The number of recommended cores are:
## Key Types
RSA keys are the most common type of keys. RSA keys of 2K length offer the best compromise between
security and performance.
> [!NOTE] RSA keys larger than 2K protect against brute force cracking and are appropriate for
> certificates that have expirations of many years. However, whenever key length is doubled, for
> example, from 2k to 4k, performance is reduced by a factor greater than 6.
ECDSA keys are also supported. In most cases, ECDSA doubles the performance of a 2K RSA key.
Supported curves are:
- secp521r1 (P-521)
- secp384r1 (P-384)
- secp256r1 (also known as prime256v1 (P-256))
## See Also
- [Deploy a Mule Application to a Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/latest/deploy-to-runtime-fabric)
- [View Runtime Fabric Health Details](https://docs.mulesoft.com/runtime-fabric/latest/view-health)
---
product: Runtime Fabric
version: 2.10
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
Watch the Anypoint Runtime Fabric Product Spotlight video to see a quick overview of Runtime Fabric.

If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat OpenShift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, the following offerings can
enable both On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE2)](https://docs.rke2.io/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/using-tkg-service-with-vsphere-supervisor.html)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS & GKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
For more details, review the
[Runtime Fabric Security Architecture](https://docs.mulesoft.com/runtime-fabric/latest/security-architecture#security-diagram) diagram.
#### Application Services
Runtime Fabric installs the following application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/2.10/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from the
following teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, the following occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

To retrieve deployable artifacts, application pods in the worker namespace must communicate with the
resource cache in the Runtime Fabric agent namespace.
### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/2.10/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/2.10/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS or GKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 2.11
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
Runtime Fabric isolates applications by running a separate Mule runtime server per application and
supports automated fail-over and horizontal scaling across multiple replicas. You can use Runtime
Fabric with managed Kubernetes services like EKS, AKS, and GKE, or with self-managed distributions
like Red Hat OpenShift and Rancher.
Watch the Anypoint Runtime Fabric Product Spotlight video to see a quick overview of Runtime Fabric.

If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat OpenShift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, these offerings can enable both
On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE2)](https://docs.rke2.io/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/using-tkg-service-with-vsphere-supervisor.html)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS & GKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
For more details, review the
[Runtime Fabric Security Architecture](https://docs.mulesoft.com/runtime-fabric/latest/security-architecture#security-diagram) diagram.
#### Application Services
Runtime Fabric installs these application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/2.11/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from these
teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, this occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

To retrieve deployable artifacts, application pods in the worker namespace must communicate with the
resource cache in the Runtime Fabric agent namespace.
### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/2.11/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/2.11/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS or GKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 2.5
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat Openshift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, the following offerings can
enable both On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE)](https://rke.docs.rancher.com/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://www.vmware.com/products/app-platform/tanzu-kubernetes-grid)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS & GKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Enable FIPS mode for security. | Yes | - | Yes | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
#### Application Services
Runtime Fabric installs the following application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
- Anypoint Monitoring sidecar: For monitoring and logging (One sidecar deployed per application
replica)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/2.5/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from the
following teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, the following occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/2.5/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/2.5/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS or GKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 2.6
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat Openshift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, the following offerings can
enable both On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE)](https://rke.docs.rancher.com/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://www.vmware.com/products/app-platform/tanzu-kubernetes-grid)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS & GKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Enable FIPS mode for security. | Yes | - | Yes | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
#### Application Services
Runtime Fabric installs the following application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
- Anypoint Monitoring sidecar: For monitoring and logging (One sidecar deployed per application
replica)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/2.6/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from the
following teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, the following occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/2.6/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/2.6/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS or GKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 2.7
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat Openshift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, the following offerings can
enable both On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE2)](https://docs.rke2.io/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/using-tkg-service-with-vsphere-supervisor.html)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS & GKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
#### Application Services
Runtime Fabric installs the following application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
- Anypoint Monitoring sidecar: For monitoring and logging (One sidecar deployed per application
replica)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/2.7/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from the
following teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, the following occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/2.7/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/2.7/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS or GKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 2.8
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat OpenShift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, the following offerings can
enable both On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE2)](https://docs.rke2.io/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/using-tkg-service-with-vsphere-supervisor.html)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS & GKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
For more details, review the
[Runtime Fabric Security Architecture](https://docs.mulesoft.com/runtime-fabric/latest/security-architecture#security-diagram) diagram.
#### Application Services
Runtime Fabric installs the following application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
- Anypoint Monitoring sidecar: For monitoring and logging (One sidecar deployed per application
replica)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/2.8/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from the
following teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, the following occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/2.8/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/2.8/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS or GKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 2.9
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat OpenShift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, the following offerings can
enable both On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE2)](https://docs.rke2.io/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/using-tkg-service-with-vsphere-supervisor.html)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS & GKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
For more details, review the
[Runtime Fabric Security Architecture](https://docs.mulesoft.com/runtime-fabric/latest/security-architecture#security-diagram) diagram.
#### Application Services
Runtime Fabric installs the following application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/2.9/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from the
following teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, the following occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

To retrieve deployable artifacts, application pods in the worker namespace must communicate with the
resource cache in the Runtime Fabric agent namespace.
### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/2.9/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/2.9/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS or GKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 3.0
is-latest-version: true
---
# Anypoint Runtime Fabric Overview
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric enables you to deploy Mule applications and API proxies to a Kubernetes
cluster that you create, configure, and manage.
Runtime Fabric isolates applications by running a separate Mule runtime server per application and
supports automated fail-over and horizontal scaling across multiple replicas. You can use Runtime
Fabric with managed Kubernetes services like EKS, AKS, GKE, and OKE, or with self-managed
distributions like Red Hat OpenShift and Rancher.
Watch the Anypoint Runtime Fabric Product Spotlight video to see a quick overview of Runtime Fabric.

If you work in public clouds and have some level of expertise in managing a Kubernetes cluster,
Runtime Fabric supports:
- [Amazon Elastics Kubernetes Service (EKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-aws-elastic-kubernetes-service/)
- [Google Kubernetes Engine (GKE)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-google-kubernetes-engine/)
- [Microsoft Azure Kubernetes Service (AKS)](https://developer.mulesoft.com/tutorials-and-howtos/runtime-fabric/runtime-fabric-azure-kubernetes-service/)
- [Oracle Kubernetes Engine (OKE)](https://www.oracle.com/cloud/cloud-native/kubernetes-engine/)
- [Red Hat OpenShift Service on AWS (ROSA)](https://aws.amazon.com/rosa/)
- [Azure Red Hat OpenShift (ARO)](https://azure.microsoft.com/en-us/products/openshift/)
- [Red Hat OpenShift Dedicated on GCP](https://cloud.google.com/blog/products/gcp/red-hats-openshift-dedicated-now-generally-available-on-google-cloud)
- [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/products/openshift)
- [Alibaba Cloud Container Service for Kubernetes (ACK)](https://www.google.com/url?q=https://www.alibabacloud.com/product/kubernetes&sa=D&source=docs&ust=1695253295782775&usg=AOvVaw1HvpOJ2m_H7JOIZzcIbyym)
If you prefer to standardize your own bare-metal solutions using public K8s distributions, depending
on your affinity to public clouds and in-house Kubernetes expertise, the following offerings can
enable both On-Prem and public cloud bare-metal solutions:
- [Amazon EKS Anywhere (EKS-A)](https://aws.amazon.com/eks/eks-anywhere/)
- [Rancher Kubernetes Engine (RKE2)](https://docs.rke2.io/)
- [Red Hat OpenShift Kubernetes engine (Entry level)](https://www.redhat.com/en/technologies/cloud-computing/openshift/kubernetes-engine)
- [Red Hat OpenShift Container Platform (OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform)
- [Red Hat OpenShift Platform Plus](https://www.redhat.com/en/technologies/cloud-computing/openshift/platform-plus)
- [VMware Tanzu Kubernetes Grid](https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/using-tkg-service-with-vsphere-supervisor.html)
Some capabilities of Anypoint Runtime Fabric include:
- Isolation between applications by running a separate Mule runtime server per application.
- Ability to run multiple versions of Mule runtime server on the same set of resources.
- Scaling applications across multiple replicas.
- Automated application fail-over.
- Application management with Anypoint Runtime Manager.
The Anypoint control plane features, Anypoint CLI, and the Mule Maven Plugin are available
universally across all partners. At runtime plane, only Helm based Runtime Fabric management and
features are supported for VMWare Tanzu, Alibaba ACK, and Rancher. The feature supportability matrix
across platforms is as follows:
| Support of following use-cases | AWS EKS and AWS EKS-A | IBM Openshift Offerings | AKS, GKE & OKE | VMWare Tanzu, Alibaba ACK, Rancher |
|---|
Access and use of Anypoint Control Plane including Runtime Manager, Access Management, Anypoint Monitoring, etc. | Yes | Yes | Yes | Yes |
Automation toolings such as the Mule Maven Plugin, Runtime Fabric APIs, and Anypoint CLI. | Yes | Yes | Yes | Yes |
Runtime Fabric functionality support for compatible K8s versions. | Yes | Yes | Yes | Yes |
Security updates for Runtime Fabric core software and Mule runtime engine images. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via helm. | Yes | Yes | Yes | Yes |
Customization of application workloads using CRDs via helm. | Yes | Yes | Yes | Yes |
Log4j support for App Monitoring | Yes | Yes | Yes | Yes |
Troubleshooting through rtfctl report command. | Yes | Yes | Yes | Yes |
Lifecycle management of Runtime Fabric software via operator. | - | Yes | - | - |
Managed lifecycle of Runtime Fabric software via control plane. | Yes | - | Yes | - |
Setup secure properties using rtfctl. | Yes | - | Yes | - |
Lifecycle management of Runtime Fabric software via rtfctl. | Yes | - | Yes | - |
Full Backup/Restore via rtfctl. | Yes | - | Yes | - |
Reference implementations (templates) - performance tuning, CRD configurations K8s best practices, recommended cluster sizing, and network configuration. | Yes | - | - | - |
## Anypoint Runtime Fabric as a Shared Responsibility
The successful operation of Anypoint Runtime Fabric is a shared responsibility. It is critical to
understand which areas you provide and manage and which areas MuleSoft provides.
The following image illustrates different MuleSoft and customer responsibilities for Anypoint
Runtime Fabric instances:

### MuleSoft Provided
MuleSoft provides the Runtime Fabric agent, Mule runtime engine (Mule), and other dependencies used
to deploy Mule applications. The Runtime Fabric agent deploys and manages Mule applications by
generating and updating Kubernetes resources such as deployments, pods, Replicasets, and ingress
resources.
#### Runtime Fabric Core Services
Anypoint Runtime Fabric and its underlying components run as Kubernetes deployment objects in EKS,
AKS, GKE, OKE, or RedHat OpenShift environments. You manage the objects via Anypoint Platform.
Linux-based operating systems are required for all nodes that run Runtime Fabric components in
Kubernetes clusters.
- The Runtime Fabric Agent
The Runtime Fabric agent is deployed as a pod in the cluster and communicates with the control
plane via an mTLS outbound connection established at startup.
The agent is event-driven. When an application state change generates a Kubernetes event, the
agent sends metadata describing the current state of the application to the control plane.
Kubernetes events include application pod starts, updates, or restarts.
The agent also listens for incoming requests from the control plane. When the agent receives a
message from the control plane, the agent makes changes to the Kubernetes resources specified in
the message. Such changes include creating a new application, updating an existing application, or
deleting an application.
- Mule cluster IP service
Provides an API that Mule applications use to discover their peers inside the cluster.
- Resource cache
Provides a cluster-local cache of application dependencies.
These services are isolated within the core Runtime Fabric installation namespace and replicas.
These services do not multiply as applications or nodes increase.
For more details, review the
[Runtime Fabric Security Architecture](https://docs.mulesoft.com/runtime-fabric/latest/security-architecture#security-diagram) diagram.
#### Application Services
Runtime Fabric installs the following application services only when they are configured by users:
- Persistence Gateway: Provides a persistent ObjectStore v2 interface to Mule applications
(Replication configured by user)
### Customer Managed
Customers are responsible for provisioning, configuring, and managing the Kubernetes cluster used
for Runtime Fabric. Additional configuration used to set up or enable capabilities on the Kubernetes
cluster, such as those listed below, are also the customer’s responsibility to manage:
- Ingress controller and [Customizations to Ingress resources](https://docs.mulesoft.com/runtime-fabric/latest/custom-ingress-configuration)
- External load balancing
- Log forwarding
- Monitoring
- Network ports, NAT gateways, and proxies
- Host runtime and networking
- Provisioning and management of the Kubernetes environment. This requires assistance from the
following teams in your organization:
- IT team to provision and manage the infrastructure
- Network team to specify allowed ports and configure proxy settings
- Security team to verify compliance and obtain security certificates
## How Application Deployments Work in Anypoint Runtime Fabric
When you deploy an application in Anypoint Runtime Fabric on, the following occurs:
1. You use Runtime Manager to trigger the application deployment.
2. The Runtime Fabric agent in the target cluster receives a request to deploy the application.
3. Runtime Fabric assigns the application to an existing namespace or creates a new namespace if
necessary.
4. Runtime Fabric generates the appropriate Kubernetes resources, including a deployment,
ConfigMap, secrets, services, and ingress.
5. If the Kubernetes deployment resource includes an init container, it fetches dependencies from
the Runtime Fabric resource cache.
6. If the resource cache doesn’t contain required dependencies, Runtime Fabric fetches them from
the control plane and adds them to the resource cache.

To retrieve deployable artifacts, application pods in the worker namespace must communicate with the
resource cache in the Runtime Fabric agent namespace.
### Assigning Namespaces in Anypoint Runtime Fabric
Each application is deployed into a Kubernetes namespace based on the application’s environment.
1. Runtime Fabric searches for a namespace with the label
`rtf.mulesoft.com/envId=`.
2. If Runtime Fabric can’t find that label, it searches for a namespace with the name
``.
3. If Runtime Fabric can’t find that namespace, it creates a new namespace called
``.

### Monitoring Application Deployments
The Runtime Fabric agent monitors Kubernetes Deployments labelled with `rtf.mulesoft.com/id`. When
Kubernetes updates the state of the deployment, the agent sends that update to the control plane.
See [Logs](https://docs.mulesoft.com/runtime-fabric/latest/limitations-self#logs) and [Monitoring](https://docs.mulesoft.com/runtime-fabric/latest/limitations-self#monitoring) for additional
information.
## Feature Support List for Runtime Fabric
The following table lists supported and non-supported features.
| Feature | Status |
|---|
Support for deploying Mules and API Gateways | Supported |
Kubernetes and Docker | Provide your instances of Kubernetes and Docker via Amazon EKS, AKS, GKE, or OKE clusters. |
Installing on any Linux distribution | Supported |
Node auto-scaling | Supported using AWS, Azure, Google Cloud, or RedHat OpenShift functionality |
External log forwarding | You can set up Log4j to forward logs to external systems. |
Internal load balancer | You must provide an internal load balancer (Ingress Controller) |
Anypoint Security Edge | Not supported |
Anypoint Security Tokenization | Not supported |
Ops Center | Not Included You can enable monitoring and alerting in AWS, Azure, Google Cloud, or RedHat OpenShift |
## Anypoint Runtime Fabric and Standalone Mule Runtimes (Hybrid Deployments)
Hybrid deployments of Mule applications require you to install a version of the Mule runtime on a
server and deploy one or more applications on the server. Each application shares the Mule runtime
server and the resources allocated to it. Other resources such as certificates or database
connections may also be shared using domains.
Anypoint Runtime Fabric provisions resources differently. Each Mule application and API gateway runs
within its own Mule runtime and in its own container. The resources available to the container are
specified when deploying a Mule application or API proxy. This enables Mule applications to
horizontally scale across nodes without relying on other dependencies. It also ensures that
different applications do not compete with each other for resources on the same node.
---
product: Runtime Fabric
version: 2.10
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Security Architecture
The following diagram illustrates the architecture of Runtime Fabric running on a customer’s
Kubernetes cluster, highlighting the communication between various Runtime Fabric core components
and the interaction with the MuleSoft Anypoint control plane. All of the outbound communications are
secured by mTLS.

## Runtime Fabric Data Flow
Review the following diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 2.11
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Security Architecture
The following diagram illustrates the architecture of Runtime Fabric running on a customer’s
Kubernetes cluster, highlighting the communication between various Runtime Fabric core components
and the interaction with the MuleSoft Anypoint control plane. All of the outbound communications are
secured by mTLS.

## Runtime Fabric Data Flow
Review this diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 2.5
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Data Flow
Review the following diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 2.6
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Data Flow
Review the following diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 2.7
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Data Flow
Review the following diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 2.8
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Security Architecture
The following diagram illustrates the architecture of Runtime Fabric running on a customer’s
Kubernetes cluster, highlighting the communication between various Runtime Fabric core components
and the interaction with the MuleSoft Anypoint control plane. All of the outbound communications are
secured by mTLS.

## Runtime Fabric Data Flow
Review the following diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 2.9
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Security Architecture
The following diagram illustrates the architecture of Runtime Fabric running on a customer’s
Kubernetes cluster, highlighting the communication between various Runtime Fabric core components
and the interaction with the MuleSoft Anypoint control plane. All of the outbound communications are
secured by mTLS.

## Runtime Fabric Data Flow
Review the following diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 3.0
is-latest-version: true
---
# Anypoint Runtime Fabric Security Architecture
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Anypoint Runtime Fabric provides a secure way to manage MuleSoft applications and API gateways
deployed onto Runtime Fabrics anywhere and ensures that you own and control the source and
destination of your data.
MuleSoft’s architecture model enables a Runtime Fabric to be installed anywhere and managed
centrally from Anypoint Platform’s cloud-based Anypoint Management Center, which includes Anypoint
Runtime Manager, Anypoint API Manager, Anypoint Monitoring and Access Management. Under this model,
all of your data flows are controlled at the location of your Runtime Fabric installation with
limited metadata and metrics stored in Anypoint Management Center. This metadata assists you with
managing, monitoring, and troubleshooting Runtime Fabrics, integration runtimes, and APIs.
This model is powered by components that run within the Runtime Fabric cluster, referred to as the
Runtime Fabric agent. The Runtime Fabric agent establishes the connection to Anypoint Management
Center. It also acts as the conduit between the internal components that power Runtime Fabric (such
as Kubernetes) and Anypoint Management Center.
## Runtime Fabric Security Architecture
The following diagram illustrates the architecture of Runtime Fabric running on a customer’s
Kubernetes cluster, highlighting the communication between various Runtime Fabric core components
and the interaction with the MuleSoft Anypoint control plane. All of the outbound communications are
secured by mTLS.

## Runtime Fabric Data Flow
Review the following diagram that shows the Runtime Fabric data flow:

### Activation Flow
The following list the steps in the activation flow process:
1. As an authorized customer, you obtain a Runtime Fabric activation token from the Runtime Fabric
Manager running on the Anypoint control plane.
2. This activation token is then presented back to Runtime Fabric Manager during creation of the
Runtime Fabric.
3. Runtime Fabric Manager responds with a TLS certificate, CA chain, and read-only credentials to
the Runtime Fabric container registry.
4. Runtime Fabric agent stores the certificate and CA chain in a Kubernetes Secret.
5. Runtime Fabric agent stores the read-only Docker registry credentials in a Kubernetes Secret.
6. Runtime Fabric agent uses the TLS certificate and CA chain to establish a mutual TLS (mTLS)
connection to Anypoint Management Center to communicate with Anypoint Runtime Manager.
7. The container registry credentials are used to pull Mule runtime images from the Runtime Fabric
container registry.
### Management Flow
The following list the steps in the management flow process:
1. The mTLS tunnel protects all metadata between Anypoint Management Center and your Runtime
Fabric. Runtime Fabric agent initiates and control the tunnel.
2. Application deployment messages contain metadata that describes the application and its
dependencies, along with links to Anypoint Management Center Configuration Resolver to fetch the
Mule app bundle and configuration from. The Configuration Resolver communication is also
protected with the same mTLS credentials.
3. You can use Anypoint Runtime Manager to manage and monitor applications in the Runtime Fabric
cluster:

- Management capabilities: start, stop, update Mule apps
- Monitoring capabilities: monitor CPU, memory, and Mule application metrics

4. You can use the Anypoint Analytics service to get insight into how your APIs are being used and
how they are performing. The service sends this metric data directly to the Analytics API over a
one-way TLS connection, see
[Mule API Analytics](https://docs.mulesoft.com/api-manager/latest/analytics-landing-page). These metrics consist of
data specific to requests:

5. Runtime Fabric agent, Mule apps, and API gateway are separate processes. If the Runtime Fabric
agent stops working or it loses connectivity with Anypoint Management Center, Mule apps and API
gateways continue to operate.
6. Mule apps and API gateways work as defined in your Mule flows, and no customer data traverse to
Anypoint Management Center. Your dataflow is specific to the application that was built and
deployed to the runtime.
### mTLS Certificate Renewal Flow
The following list the mTLS certificate renewal flow process:
1. When the mTLS certificate is close to its expiration date, the Runtime Fabric agent requests a
renewed certificate from Anypoint Management Center via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new TLS certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Runtime Fabric agent restarts and re-establishes the mTLS tunnel with the new certificate.
### Anypoint Monitoring Certificate Renewal Flow
The following list the Monitoring certificate renewal flow process:
1. When the Anypoint Monitoring certificate is close to its expiration date, Runtime Fabric agent
requests a renewed certificate from Anypoint Management Center, via the existing mTLS tunnel.
2. Anypoint Management Center responds with a new certificate.
3. Runtime Fabric agent overwrites the existing certificate with the new certificate.
4. Anypoint Monitoring containers re-establish the connection with the new certificate.
### MuleSoft Image Management
The following list the image management process:
- Vulnerability scanning and Sonar quality gates during build process.
- All newer image pulls are triggered by customer only.
- MuleSoft remediates container images and applies available patches according to the normal
remediation cadence.
- Critical vulnerabilities within 30 days
- High vulnerabilities within 90 days
- Medium vulnerabilities within 180 days
- Low vulnerabilities within 360 days
### Container Privilege Management
The following list the container privilege management process:
1. None of the containers need to run as root in Runtime Fabric.
2. Extremely pared down utilities inside the containers.
3. No Kubernetes service account attached to any application containers.
The Runtime Fabric agent and Mule cluster IP service interact with Kubernetes control plane. To
perform any action, these components need all permissions on all resources in the cluster.
## Metadata Types Stored in The Cloud
The following metadata types are stored in the cloud:
- Runtime Fabric
Information about a Runtime Fabric cluster. Specifically:
- Name
- Region
- Cluster version
- Runtime Fabric agent version
- Associated environments
- Nodes, CPU, memory capacity, Kubernetes version, docker version, Kubernetes health warnings
- Mule apps
The flow architecture that you create using Anypoint Studio. This flow architecture is metadata
about your APIs, and you upload it to Runtime Manager. Mule apps are then deployed to the
specified Runtime Fabric.
- Monitoring metrics (optional)
Metrics about the basic runtime status that are collected on a poll configured on the runtime.
Specifically:
- CPU Usage - Past Hour, Current Time
- Memory Usage - Past Hour, Current Time
- Memory Total - Past Hour, Current Time
- Message Count - Past Hour, Current Time
- Response Time - Past Hour, Current Time
- Error Count - Past Hour, Current Time
- Analytics metrics (optional)
You can enable analytics metrics to get insight into how your APIs are used and how they perform.
You can configure these metrics and store the results for a maximum of 90 days. You can capture
these metrics:
- API name
- API version
- Application
- Browser
- City
- Client IP
- Continent
- Country
- Hardware platform
- OS family
- OS major Version
- OS minor Version
- OS version
- Postal code
- Resource path
- Status code
- Timezone
- User agent type
- User agent version
- Verb
- Violated policy name
- Advanced capabilities
You can use advanced capabilities in this model to send payload information to the cloud. These
are not turned on by default and require advanced configuration to turn on and use. An example of
this capability is [Runtime Manager Insight](https://docs.mulesoft.com/runtime-manager/insight)
[https://docs.mulesoft.com/runtime-manager/insight](https://docs.mulesoft.com/runtime-manager/insight).
---
product: Runtime Fabric
version: 2.10
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of the following messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.10/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 2.11
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of these messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.11/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 2.5
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of the following messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.5/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 2.6
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of the following messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.6/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 2.7
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of the following messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.7/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 2.8
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of the following messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.8/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 2.9
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of the following messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.9/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 3.0
is-latest-version: true
---
# Associating Business Groups and Environments to Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Before deploying applications or API gateways to Anypoint Runtime Fabric, you must associate a
Runtime Fabric with one or more Anypoint Platform environments. When deploying a Mule application,
you cannot select a Runtime Fabric as a deployment target until you have performed this step in the
environment where you want to deploy.
> [!NOTE] Runtime Fabric appliance does not support this feature.
## Associate Business Groups
You can share Runtime Fabric with an organization’s business groups. This allows other business
groups to deploy applications to a single Runtime Fabric. To use this feature, the business groups
must fall under the scope of the organization the Runtime Fabric is registered to.
To associate a Runtime Fabric across business groups:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** tab on the left sidebar.
3. Select the Runtime Fabric to manage.
4. On the **Environments** tab, click **Add business group**.
5. Select the business groups from the drop-down list:
- **This business group and its children**
This option allows you to select either all environments, all sandbox environment or all
production environments for the selected business group and all its children.
- **Selected business groups**
Enter the business group name in the **Add business group** field and select it from the list.
This option allows you to select specific environments for root organizations as well as it’s
children.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

## Associate Environments
To associate an environment to a Runtime Fabric:
1. From Anypoint Platform, select **Runtime Manager**.
2. Select the **Runtime Fabric** section on the left sidebar.
3. Select the Runtime Fabric based on the name used during installation.
4. Click the **Environments** tab.
5. Select the environments from the drop-down list:
- **All Environment**
Associates the Runtime Fabric with all sandbox and production environments in the business
group and all its children, or in specific child business organizations.
- **All Sandbox Environments** Associates the Runtime Fabric with only sandbox environments in
the business group and all its children, or in specific child business organizations.
- **All Production Environments**
Associates the Runtime Fabric with only production environments in the business group and all
its children, or in specific child business organizations.
- **Select Environments Manually**
Allows you to associate the Runtime Fabric with specific environments in specific child
business groups.
- Check the **Sandbox** or **Production** box to bulk select all sandbox or production
environments in specific child business organizations.
- Check the box for each specific subenvironment to individually select the desired
environments in specific child business organizations.
6. Click **Save Changes** or **Discard Changes** to discard or confirm the association.

### Production Environments
When selecting a production environment, a dialog is displayed that verifies Runtime Fabric has been
configured to operate in a production setting. This includes proper hardware configuration, log
forwarding, alerting, and possible failure modes.
If your Runtime Fabric is running with an unsupported configuration or does not meet the
requirements for a production environment, one of the following messages is displayed:
- `This Runtime Fabric is not supported for any workloads. Learn more.`
- `This Runtime Fabric is suitable for non-production workloads only. Learn more.`
Refer to [Installing Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/latest/install-index) for additional information.
Proper configuration must be complete before operating and deploying production applications on
Runtime Fabric.
---
product: Runtime Fabric
version: 2.10
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use the following procedures to back up and restore Runtime Fabric services and the required state
necessary for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
Starting with `rtfctl` version 1.0.70, if you use an rtf-agent version higher than 2.6.52, the
`rtfctl` backup bundle does not contain the Mule app deployments. The deployments are synched from
the Runtime Manager UI at the time of restoration.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, the following is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, the following is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
Any changes you make after you perform a backup will not be present if you restore the cluster from
that backup. For example, if you perform a backup, delete an application, and then restore from that
backup, the application you deleted will be recreated in the cluster, but it will be marked as
deleted in Runtime Manager. As a result the application’s status will show as **applying** in
Runtime Manager.
Runtime Fabric does not restore the following control plane changes if you make them after you take
a backup:
- Created apps
- Delete apps
- Updated app configurations
- Agent upgrades
You need to manually resolve such inconsistencies by reapplying the changes you made after
performing the backup, as these changes can’t be reapplied from the control plane.
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run the following command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review the following information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In the following example, you have a Runtime Fabric with a set of Mule applications and
configurations deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a
backup and perform a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running the following command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
## See Also
- [Hardening Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.10/hardening-runtime-fabric)
---
product: Runtime Fabric
version: 2.11
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use these procedures to back up and restore Runtime Fabric services and the required state necessary
for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
Starting with `rtfctl` version 1.0.70, if you use an rtf-agent version higher than 2.6.52, the
`rtfctl` backup bundle does not contain the Mule app deployments. The deployments are synched from
the Runtime Manager UI at the time of restoration.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, this is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, this is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
Any changes you make after you perform a backup will not be present if you restore the cluster from
that backup. For example, if you perform a backup, delete an application, and then restore from that
backup, the application you deleted will be recreated in the cluster, but it will be marked as
deleted in Runtime Manager. As a result the application’s status will show as **applying** in
Runtime Manager.
Runtime Fabric does not restore these control plane changes if you make them after you take a
backup:
- Created apps
- Delete apps
- Updated app configurations
- Agent upgrades
You need to manually resolve such inconsistencies by reapplying the changes you made after
performing the backup, as these changes can’t be reapplied from the control plane.
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run this command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review this information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In this example, you have a Runtime Fabric with a set of Mule applications and configurations
deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a backup and perform
a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running this command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
\[\[backup-restore-rtfctl\]
## Commands for Backing Up and Restoring a Runtime Fabric with RTFCTL
Use these commands to back up and restore a Runtime Fabric.
| Command | Description |
|---|
| Performs a back up of the local Runtime Fabric state |
| Restores local Runtime Fabric state from a backup |
### `backup`
Performs a back up of the local Runtime Fabric state.
rtfctl backup .tar.gz --namespace
### `restore`
Restores the local Runtime Fabric state from a backup
rtfctl restore .tar.gz --namespace
Restore only the core Runtime Fabric software components without triggering application
synchronization from the control plane.
rtfctl restore .tar.gz --namespace --core-software-only
## See Also
- [Hardening Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.11/hardening-runtime-fabric)
---
product: Runtime Fabric
version: 2.5
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use the following procedures to back up and restore Runtime Fabric services and the required state
necessary for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, the following is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, the following is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
Any changes you make after you perform a backup will not be present if you restore the cluster from
that backup. For example, if you perform a backup, delete an application, and then restore from that
backup, the application you deleted will be recreated in the cluster, but it will be marked as
deleted in Runtime Manager. As a result the application’s status will show as **applying** in
Runtime Manager.
Runtime Fabric does not restore the following control plane changes if you make them after you take
a backup:
- Created apps
- Delete apps
- Updated app configurations
- Agent upgrades
You need to manually resolve such inconsistencies by reapplying the changes you made after
performing the backup, as these changes can’t be reapplied from the control plane.
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run the following command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review the following information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In the following example, you have a Runtime Fabric with a set of Mule applications and
configurations deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a
backup and perform a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running the following command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
---
product: Runtime Fabric
version: 2.6
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use the following procedures to back up and restore Runtime Fabric services and the required state
necessary for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
Starting with `rtfctl` version 1.0.70, if you use an rtf-agent version higher than 2.6.52, the
`rtfctl` backup bundle does not contain the Mule app deployments. The deployments are synced from
the Runtime Manager UI at the time of restoration.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, the following is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, the following is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
All changes performed on the control plane side don’t need a new backup to be taken into account
because those are taken from the platform at the time of restoration.
Any changes done on the cluster side aren’t part of the restoration and must be reapplied. These
changes include:
- RTFCTL managed secure properties
- Ingress resources on Runtime Fabric namespace
- ConfigMap settings
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run the following command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review the following information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In the following example, you have a Runtime Fabric with a set of Mule applications and
configurations deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a
backup and perform a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running the following command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
## See Also
- [Hardening Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.6/hardening-runtime-fabric)
---
product: Runtime Fabric
version: 2.7
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use the following procedures to back up and restore Runtime Fabric services and the required state
necessary for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
Starting with `rtfctl` version 1.0.70, if you use an rtf-agent version higher than 2.6.52, the
`rtfctl` backup bundle does not contain the Mule app deployments. The deployments are synced from
the Runtime Manager UI at the time of restoration.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, the following is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, the following is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
All changes performed on the control plane side don’t need a new backup to be taken into account
because those are taken from the platform at the time of restoration.
Any changes done on the cluster side aren’t part of the restoration and must be reapplied. These
changes include:
- Agent upgrades
- RTFCTL managed secure properties
- Ingress resources on Runtime Fabric namespace
- ConfigMap settings
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run the following command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review the following information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In the following example, you have a Runtime Fabric with a set of Mule applications and
configurations deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a
backup and perform a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running the following command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
## See Also
- [Hardening Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.7/hardening-runtime-fabric)
---
product: Runtime Fabric
version: 2.8
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use the following procedures to back up and restore Runtime Fabric services and the required state
necessary for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
Starting with `rtfctl` version 1.0.70, if you use an rtf-agent version higher than 2.6.52, the
`rtfctl` backup bundle does not contain the Mule app deployments. The deployments are synced from
the Runtime Manager UI at the time of restoration.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, the following is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, the following is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
All changes performed on the control plane side don’t need a new backup to be taken into account
because those are taken from the platform at the time of restoration.
Any changes done on the cluster side aren’t part of the restoration and must be reapplied. These
changes include:
- RTFCTL managed secure properties
- Ingress resources on Runtime Fabric namespace
- ConfigMap settings
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run the following command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review the following information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In the following example, you have a Runtime Fabric with a set of Mule applications and
configurations deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a
backup and perform a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running the following command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
## See Also
- [Hardening Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.8/hardening-runtime-fabric)
---
product: Runtime Fabric
version: 2.9
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use the following procedures to back up and restore Runtime Fabric services and the required state
necessary for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
Starting with `rtfctl` version 1.0.70, if you use an rtf-agent version higher than 2.6.52, the
`rtfctl` backup bundle does not contain the Mule app deployments. The deployments are synched from
the Runtime Manager UI at the time of restoration.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, the following is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, the following is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
Any changes you make after you perform a backup will not be present if you restore the cluster from
that backup. For example, if you perform a backup, delete an application, and then restore from that
backup, the application you deleted will be recreated in the cluster, but it will be marked as
deleted in Runtime Manager. As a result the application’s status will show as **applying** in
Runtime Manager.
Runtime Fabric does not restore the following control plane changes if you make them after you take
a backup:
- Created apps
- Delete apps
- Updated app configurations
- Agent upgrades
You need to manually resolve such inconsistencies by reapplying the changes you made after
performing the backup, as these changes can’t be reapplied from the control plane.
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run the following command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review the following information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In the following example, you have a Runtime Fabric with a set of Mule applications and
configurations deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a
backup and perform a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running the following command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
## See Also
- [Hardening Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/2.9/hardening-runtime-fabric)
---
product: Runtime Fabric
version: 3.0
is-latest-version: true
---
# Backing Up and Restoring Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Use the following procedures to back up and restore Runtime Fabric services and the required state
necessary for all deployed applications and their configurations.
Data about the state of Runtime Fabric is distributed across the Kubernetes cluster. To preserve
applications in the event of system failure, confirm that Runtime Fabric is automatically backed up.
When scheduling backups, follow best practices by configuring hourly backups. Store your backup on
external storage outside the Runtime Fabric cluster.
Starting with `rtfctl` version 1.0.70, if you use an rtf-agent version higher than 2.6.52, the
`rtfctl` backup bundle does not contain the Mule app deployments. The deployments are synched from
the Runtime Manager UI at the time of restoration.
## When to Use the Back Up and Restore Process
Common scenarios for using the backup and restore process include:
- Performing a major infrastructure upgrade, such as migrating nodes in a Runtime Fabric cluster to
a new version of an operating system.
- Migrating your Runtime Fabric cluster from on-premises to the cloud or migrating from one cloud
provider to another.
- Performing a failover to a standby cluster.
> [!WARNING] Do not use the backup and restore process to create concurrent running instances of
> Runtime Fabric. In other words, do not back up Runtime Fabric A, use that back up to create a new
> Runtime Fabric B while Runtime Fabric A is still active, and attempt to run them concurrently.
> This creates application deployment issues.
## What Gets Backed Up
The Runtime Fabric backup process affects only the Runtime Fabric cluster. The backup process does
not back up data from the control plane.
For regular Kubernetes clusters, the following is backed up:
- Runtime Fabric cluster details, such as controller and worker node configurations
- Cluster-level secure properties
- All data in the Runtime Fabric and application namespace, such as data about deployments, services
and service accounts, secrets, ingress resources, role bindings, cluster role configurations, and
ConfigMap settings
- The current `rtfctl` version
- The current Runtime Fabric agent version
- Application details including:
- Application configurations for pods, replicasets, application metadata, and templates
- Application properties configured in Runtime Manager
- Data from other Mulesoft configurations, including:
- Object Store
- Edge
- Anypoint Monitoring and external logforwarders
For OpenShift clusters, the following is backed up:
- Custom Resources
- Application namespaces and for each one:
- Secrets
- Service Account
- ConfigMap
- Services
- CronJobs
- DaemonSets
- Deployments
- Ingress
- Role
- Role Bindings
- Ingress from Runtime Fabric namespace
## What Does Not Get Backed Up
- Data about application state
- Data from Ops Center
- Data from the control plane
### Changes Made After Performing a Backup Are Not Restored
Any changes you make after you perform a backup will not be present if you restore the cluster from
that backup. For example, if you perform a backup, delete an application, and then restore from that
backup, the application you deleted will be recreated in the cluster, but it will be marked as
deleted in Runtime Manager. As a result the application’s status will show as **applying** in
Runtime Manager.
Runtime Fabric does not restore the following control plane changes if you make them after you take
a backup:
- Created apps
- Delete apps
- Updated app configurations
- Agent upgrades
You need to manually resolve such inconsistencies by reapplying the changes you made after
performing the backup, as these changes can’t be reapplied from the control plane.
## Create a Backup
Before creating a backup:
- Verify that Runtime Fabric is installed and running successfully.
- Verify that `rtfctl` command-line tool is upgraded to version 0.3.102 or later. Verify the version
by running `rtfctl version` on any Runtime Fabric node.
- Confirm that the `rtfctl` binary is present in the current directory and `kubectl` is present in
the user `$PATH`. Additionally, confirm that you’ve set the Kubernetes (`kubectl`) context to the
intended backup cluster.
> [!WARNING] Creating a backup per node, such as via a VM snapshot, is not supported.
To create a backup, run the following command:
```copy
./rtfctl backup
```
This command creates a backup of the current system state in ``, which can be
any path in the file system that you have write access to, such as:
`/opt/anypoint/runtimefabric/backup.tar.gz`.
## Perform a Restore
> [!IMPORTANT] Before you restore a cluster, review the following information:
>
> - You must perform the restore on the same Runtime Fabric cluster that is listed in Runtime
> Manager (the same cluster in the control plane).
>
> - Configuration changes you make to deployed applications and management services after a back up
> are not restored.
>
> - Application monitoring metrics are not restored.
>
> - When restoring on an existing Runtime Fabric cluster, use the same version of the `rftctl`
> command-line utility that you used to create the backup.
Runtime Fabric provides two target options when restoring a cluster from a backup:
- Use an existing Runtime Fabric cluster.
- Create a new Kubernetes cluster with the same configuration as the backed-up cluster. This
includes the same number of servers, disks, etc.
To restore a cluster on Runtime Fabric, create a cluster **without** running the `rtfctl` install
command. After you create the cluster, complete the restore process.
1. Choose a target option for restoring a cluster.
2. Ensure you have installed `rftctl` in the cluster.
3. Copy the backup file you previously created, and make sure it is available to `rtfctl`.
4. Confirm your Kubernetes (kubectl) context is set to the backed-up cluster, and scale down all
Runtime Fabric components on the original backed-up cluster:
```copy
kubectl scale --replicas=0 -n rtf deployment.apps/agent
```
5. Restore the cluster from the backup:
./rtfctl restore
This process may require several minutes to complete.
For Runtime Fabric, confirm the `rtfctl` binary is present in the current directory and the
Kubernetes (kubectl) context is set to the cluster you are restoring to.
## Create a Backup and Restore for OpenShift Clusters
In the following example, you have a Runtime Fabric with a set of Mule applications and
configurations deployed on cluster (named `cluster-1`) inside an OpenShift cluster. To create a
backup and perform a restore, follow these steps:
1. Run the backup command:
```copy
./rtfctl backup -n
```
> [!NOTE] MuleSoft performs the back up based on the detected cluster you have. Though, you can
> also manually specify the type of back up to perform using the flag `--backup-type` string.
> The flag options are `auto`, `openshift`, and `full`. With the default `auto` flag option,
> MuleSoft automatically detects the infrastructure cluster provider to perform the back up.
2. Shutdown `cluster-1`.
3. Delete all Mule application deployments by running the following command:
```copy
kubectl delete ns
```
Do not remove applications from the Runtime Manager UI.
4. Remove the Runtime Fabric installation.
5. Remove the Runtime Fabric operator from the OpenShift cluster (optional).
6. Send a `curl` command to the management plane API endpoint to regenerate the activation ID.
The Runtime Fabric shows as ready for activation. You can install and activate Runtime Fabric
again on a new OpenShift cluster.
```copy
curl --location --request PATCH 'https://anypoint.mulesoft.com/runtimefabric/api/organizations/{org-id}/fabrics/{fabric-id}/regenerateActivationData' \
--header 'Authorization: Bearer {bearer-token}
```
7. Install the Runtime Fabric operator on `cluster-2` using the regenerated activation data ID.
The installation must be on the same Runtime Fabric namespace. All labels on the backup point to
that installation.
8. Run the restore command to pick the set of resources and restore them in the cluster:
```copy
./rtfctl restore -n
```
9. Check the status of your Runtime Fabric deployments.
\[\[backup-restore-rtfctl\]
## Commands for Backing Up and Restoring a Runtime Fabric with RTFCTL
Use these commands to back up and restore a Runtime Fabric.
| Command | Description |
|---|
| Performs a back up of the local Runtime Fabric state |
| Restores local Runtime Fabric state from a backup |
### `backup`
Performs a back up of the local Runtime Fabric state.
rtfctl backup .tar.gz --namespace
### `restore`
Restores the local Runtime Fabric state from a backup
rtfctl restore .tar.gz --namespace
Restore only the core Runtime Fabric software components without triggering application
synchronization from the control plane.
rtfctl restore .tar.gz --namespace --core-software-only
## See Also
- [Hardening Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/latest/hardening-runtime-fabric)
---
product: Runtime Fabric
version: 2.10
---
# Cluster Configurations
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
To use the complete value of Runtime Fabric, refer to your respective Kubernetes (K8s) provider’s
configuration specification. To get started, use the following recommended configurations to set up
your cluster.
| Type of configuration | EKS-A | EKS | Appliance | MuleSoft requirements |
|---|
Infrastructure | Vsphere/Bare Metal | AWS | Pre-provisioned hosts | - |
Ingress | Emissary, Nginx-ingress or any other available ingress options. | Nginx ingress | Nginx ingress | Customer-managed component config |
Load balancer | MetalLB | ELB | - | Customer-managed component config |
Monitoring | Grafana/Prometheus | CloudWatch | Grafana/Prometheus | Check Requirements and Limitations of Runtime Fabric |
Local cluster registry | Harbor or any other compliant implementation | ECR | Docker registry running inside the cluster | |
Cluster size limits | Kubernetes imposed limits on the cluster size. | 1000 managed nodes | 21 total (5 control plane nodes + 16 worker nodes | - |
CK8s distro | EKS-D | EKS-D | Custom distro (planet) | Check Requirements and Limitations of Runtime Fabric |
Cluster mgmt tool | eksctl or eksctl-anywhere
| eksctl
| Custom utility (rtfctl) | Check Requirements and Limitations of Runtime Fabric |
---
product: Runtime Fabric
version: 2.11
---
# Cluster Configurations
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
To use the complete value of Runtime Fabric, refer to your respective Kubernetes (K8s) provider’s
configuration specification. To get started, use these recommended configurations to set up your
cluster.
| Type of configuration | EKS-A | EKS | Appliance | MuleSoft requirements |
|---|
Infrastructure | Vsphere/Bare Metal | AWS | Pre-provisioned hosts | - |
Ingress | Emissary, Nginx-ingress or any other available ingress options. | Nginx ingress | Nginx ingress | Customer-managed component config |
Load balancer | MetalLB | ELB | - | Customer-managed component config |
Monitoring | Grafana/Prometheus | CloudWatch | Grafana/Prometheus | Check Requirements and Limitations of Runtime Fabric |
Local cluster registry | Harbor or any other compliant implementation | ECR | Docker registry running inside the cluster | |
Cluster size limits | Kubernetes imposed limits on the cluster size. | 1000 managed nodes | 21 total (5 control plane nodes + 16 worker nodes | - |
CK8s distro | EKS-D | EKS-D | Custom distro (planet) | Check Requirements and Limitations of Runtime Fabric |
Cluster mgmt tool | eksctl or eksctl-anywhere
| eksctl
| Custom utility (rtfctl) | Check Requirements and Limitations of Runtime Fabric |
---
product: Runtime Fabric
version: 2.5
---
# Cluster Configurations
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
To use the complete value of Runtime Fabric, refer to your respective Kubernetes (K8s) provider’s
configuration specification. To get started, use the following recommended configurations to set up
your cluster.
| Type of configuration | EKS-A | EKS | Appliance | MuleSoft requirements |
|---|
Infrastructure | Vsphere/Bare Metal | AWS | Pre-provisioned hosts | - |
Ingress | Emissary, Nginx-ingress or any other available ingress options. | Nginx ingress | Nginx ingress | Customer-managed component config |
Load balancer | MetalLB | ELB | - | Customer-managed component config |
Monitoring | Grafana/Prometheus | CloudWatch | Grafana/Prometheus | Check Requirements and Limitations of Runtime Fabric |
Local cluster registry | Harbor or any other compliant implementation | ECR | Docker registry running inside the cluster | |
Cluster size limits | Kubernetes imposed limits on the cluster size. | 1000 managed nodes | 21 total (5 control plane nodes + 16 worker nodes | - |
CK8s distro | EKS-D | EKS-D | Custom distro (planet) | Check Requirements and Limitations of Runtime Fabric |
Cluster mgmt tool | eksctl or eksctl-anywhere
| eksctl
| Custom utility (rtfctl) | Check Requirements and Limitations of Runtime Fabric |
---
product: Runtime Fabric
version: 2.6
---
# Cluster Configurations
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
To use the complete value of Runtime Fabric, refer to your respective Kubernetes (K8s) provider’s
configuration specification. To get started, use the following recommended configurations to set up
your cluster.
| Type of configuration | EKS-A | EKS | Appliance | MuleSoft requirements |
|---|
Infrastructure | Vsphere/Bare Metal | AWS | Pre-provisioned hosts | - |
Ingress | Emissary, Nginx-ingress or any other available ingress options. | Nginx ingress | Nginx ingress | Customer-managed component config |
Load balancer | MetalLB | ELB | - | Customer-managed component config |
Monitoring | Grafana/Prometheus | CloudWatch | Grafana/Prometheus | Check Requirements and Limitations of Runtime Fabric |
Local cluster registry | Harbor or any other compliant implementation | ECR | Docker registry running inside the cluster | |
Cluster size limits | Kubernetes imposed limits on the cluster size. | 1000 managed nodes | 21 total (5 control plane nodes + 16 worker nodes | - |
CK8s distro | EKS-D | EKS-D | Custom distro (planet) | Check Requirements and Limitations of Runtime Fabric |
Cluster mgmt tool | eksctl or eksctl-anywhere
| eksctl
| Custom utility (rtfctl) | Check Requirements and Limitations of Runtime Fabric |
---
product: Runtime Fabric
version: 2.7
---
# Cluster Configurations
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
To use the complete value of Runtime Fabric, refer to your respective Kubernetes (K8s) provider’s
configuration specification. To get started, use the following recommended configurations to set up
your cluster.
| Type of configuration | EKS-A | EKS | Appliance | MuleSoft requirements |
|---|
Infrastructure | Vsphere/Bare Metal | AWS | Pre-provisioned hosts | - |
Ingress | Emissary, Nginx-ingress or any other available ingress options. | Nginx ingress | Nginx ingress | Customer-managed component config |
Load balancer | MetalLB | ELB | - | Customer-managed component config |
Monitoring | Grafana/Prometheus | CloudWatch | Grafana/Prometheus | Check Requirements and Limitations of Runtime Fabric |
Local cluster registry | Harbor or any other compliant implementation | ECR | Docker registry running inside the cluster | |
Cluster size limits | Kubernetes imposed limits on the cluster size. | 1000 managed nodes | 21 total (5 control plane nodes + 16 worker nodes | - |
CK8s distro | EKS-D | EKS-D | Custom distro (planet) | Check Requirements and Limitations of Runtime Fabric |
Cluster mgmt tool | eksctl or eksctl-anywhere
| eksctl
| Custom utility (rtfctl) | Check Requirements and Limitations of Runtime Fabric |
---
product: Runtime Fabric
version: 2.8
---
# Cluster Configurations
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
To use the complete value of Runtime Fabric, refer to your respective Kubernetes (K8s) provider’s
configuration specification. To get started, use the following recommended configurations to set up
your cluster.
| Type of configuration | EKS-A | EKS | Appliance | MuleSoft requirements |
|---|
Infrastructure | Vsphere/Bare Metal | AWS | Pre-provisioned hosts | - |
Ingress | Emissary, Nginx-ingress or any other available ingress options. | Nginx ingress | Nginx ingress | Customer-managed component config |
Load balancer | MetalLB | ELB | - | Customer-managed component config |
Monitoring | Grafana/Prometheus | CloudWatch | Grafana/Prometheus | Check Requirements and Limitations of Runtime Fabric |
Local cluster registry | Harbor or any other compliant implementation | ECR | Docker registry running inside the cluster | |
Cluster size limits | Kubernetes imposed limits on the cluster size. | 1000 managed nodes | 21 total (5 control plane nodes + 16 worker nodes | - |
CK8s distro | EKS-D | EKS-D | Custom distro (planet) | Check Requirements and Limitations of Runtime Fabric |
Cluster mgmt tool | eksctl or eksctl-anywhere
| eksctl
| Custom utility (rtfctl) | Check Requirements and Limitations of Runtime Fabric |
---
product: Runtime Fabric
version: 2.9
---
# Cluster Configurations
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
To use the complete value of Runtime Fabric, refer to your respective Kubernetes (K8s) provider’s
configuration specification. To get started, use the following recommended configurations to set up
your cluster.
| Type of configuration | EKS-A | EKS | Appliance | MuleSoft requirements |
|---|
Infrastructure | Vsphere/Bare Metal | AWS | Pre-provisioned hosts | - |
Ingress | Emissary, Nginx-ingress or any other available ingress options. | Nginx ingress | Nginx ingress | Customer-managed component config |
Load balancer | MetalLB | ELB | - | Customer-managed component config |
Monitoring | Grafana/Prometheus | CloudWatch | Grafana/Prometheus | Check Requirements and Limitations of Runtime Fabric |
Local cluster registry | Harbor or any other compliant implementation | ECR | Docker registry running inside the cluster | |
Cluster size limits | Kubernetes imposed limits on the cluster size. | 1000 managed nodes | 21 total (5 control plane nodes + 16 worker nodes | - |
CK8s distro | EKS-D | EKS-D | Custom distro (planet) | Check Requirements and Limitations of Runtime Fabric |
Cluster mgmt tool | eksctl or eksctl-anywhere
| eksctl
| Custom utility (rtfctl) | Check Requirements and Limitations of Runtime Fabric |
---
product: Runtime Fabric
version: 3.0
is-latest-version: true
---
# Configure a Local Registry for Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
By default, Runtime Fabric pulls the Docker images it needs for installation, management, and
application deployments from the MuleSoft-managed registry. If your security policies require that
all images come from a registry you control, you can configure Runtime Fabric to use a local
registry instead.
## Requirements and Considerations
You can configure a local registry across all Runtime Fabric supported platform vendors to store
Mule Docker images and Runtime Fabric software images.
> [!IMPORTANT] You cannot use a local registry for installation or management of the Runtime Fabric
> operator on OpenShift.
### Supported Configurations
- You can use [Runtime Fabric command line utility (`rtfctl`)](https://docs.mulesoft.com/runtime-fabric/latest/install-rtfctl) version 0.3.150 or
later. `rtfctl` installs Runtime Fabric agent to the latest version by default.
- You can use Helm to install Runtime Fabric and configure your local registry.
- You can use a local registry only if you create a new Runtime Fabric instance of version 1.12.0 or
later. Once configured, you can upgrade to later versions without reinstalling Runtime Fabric.
To add a local registry to an existing Runtime Fabric instance, ensure the installation is
Helm-based (see [Migrating to Helm Managed Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/latest/migrate-helm)) and follow the steps in
this page.
> [!IMPORTANT] Local registry configuration is a one-way change. After you configure Runtime Fabric
> to use a local registry, you cannot reconfigure it to use the `rtf-runtime-registry` endpoint. To
> revert, you must install a new Runtime Fabric instance using the standard installation procedure.
### Synchronization and Authentication
You must keep your local registry synchronized with the images Runtime Fabric requires:
- [Synchronize Runtime Fabric Core Software Images](#synch-rtf) — synchronize automatically or
manually before installation and upgrades.
- [Synchronize Mule Runtime Images](#synch-runtime) — synchronize manually for each runtime version
you deploy.
- Third-party dependencies required by your applications — MuleSoft does not provide tooling for
this. Push all required images to your local registry before starting deployments.
If your local registry requires authentication, pull secret propagation across namespaces is your
responsibility. Alternatively, Runtime Fabric can synchronize your pull secret automatically if you
apply this label to the secret:
labels: rtf.mulesoft.com/synchronized: "true"
Ensure that the
[corresponding credentials are configured as a Kubernetes secret in the `rtf` namespace](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#registry-secret-existing-credentials).
You need the name of this secret when you install Runtime Fabric.
## Synchronize Images to Your Local Registry
Before you can install or configure Runtime Fabric with a local registry, you must synchronize the
required images. These sections cover Runtime Fabric core software images and Mule runtime images
separately, as they use different sources and synchronization methods.
### Synchronize Runtime Fabric Core Software Images
To install Runtime Fabric, you must first synchronize the core software images to your local
registry. You can perform the synchronization by either running a script automatically or executing
commands manually.
- Synchronize Automatically
- Synchronize Manually
To synchronize the dependencies images automatically, run a bash script that uses the `docker`,
`curl`, and `jq` tools.
> [!NOTE] Automated registry synchronization is available from Runtime Fabric version 2.6.0 onwards.
> Previous dependencies cannot be retrieved using this script.
Ensure that your
[Access Management permission for Runtime Manager](https://docs.mulesoft.com/access-management/permissions-by-product#runtime-manager)
is set to **Read Runtime Fabric**, which enables you to query Runtime Fabrics in your organization.
1. In your Unix shell, run this script, replacing `agent-version`, `authorization-token`,
`mulesoft-docker-server`, and any other parameter value where appropriate:
For non-OpenShift environments:
```copy
##!/usr/bin/env bash
dependencies=$(curl /runtimefabric/api/agentmanifests/ -H 'Authorization: bearer ' | jq -c '.dependencies |.[] | select(.provider | contains("generic")) | "\(.artifact):\(.version)"' | sed 's/"//g')
for i in $dependencies
do
echo "Processing $i"
docker pull //$i
docker tag //$i /mulesoft/$i
docker push /mulesoft/$i
done
```
For OpenShift environments:
```copy
##!/usr/bin/env bash
dependencies=$(curl /runtimefabric/api/agentmanifests/ -H 'Authorization: bearer ' | jq -c '.dependencies |.[] | select(.provider | contains("openshift")) | "\(.artifact):\(.version)"' | sed 's/"//g')
for i in $dependencies
do
echo "Processing $i"
docker pull //$i
docker tag //$i /mulesoft/$i
docker push /mulesoft/$i
done
```
Table 1. Script Parameters | Parameters | Description |
|---|
platform-url
| Anypoint Platform URL, such as https://anypoint.mulesoft.com, or https://eu1.anypoint.mulesoft.com. |
agent-version
| Runtime Fabric agent release version required for the synchronization. |
authorization-token
| Authorization token required for using connected apps to perform the synchronization. When you use this token, a link redirects to the connected apps page. |
mulesoft-docker-server
| MuleSoft Docker server required for the synchronization. US Cloud: rtf-runtime-registry.kprod.msap.io EU Cloud: rtf-runtime-registry.kprod-eu.msap.io Canada Cloud: rtf-runtime-registry-mulesoft-cp.sfdc-58ktaz.svc.sfdcfc.net Japan Cloud: rtf-runtime-registry-mulesoft-cp.sfdc-mchho0.svc.sfdcfc.net India Cloud: rtf-runtime-registry-mulesoft-cp.sfdc-y37hzm.svc.sfdcfc.net Australia Cloud: rtf-runtime-registry-mulesoft-cp.sfdc-vwfla6.svc.sfdcfc.net United Kingdom Cloud: rtf-runtime-registry-mulesoft-cp.sfdc-5pakla.svc.sfdcfc.net
|
image-path
| Image namespace path on the registry. US Cloud and EU Cloud: mulesoft Canada Cloud, Japan Cloud, India Cloud, Australia Cloud, and United Kingdom Cloud: sfci/mulesoft/ms-docker-image
|
> [!NOTE] For Canada Cloud, Japan Cloud, India Cloud, Australia Cloud, and United Kingdom Cloud,
> images in the SFCI registry use an `ms-` prefix in the artifact name (for example, `ms-rtf-agent`
> instead of `rtf-agent`). The automated script does not account for this prefix difference, so you
> must manually prepend `ms-` to each artifact name when pulling from the SFCI registry for these
> regions.
To synchronize the dependencies images manually, follow these steps:
> [!NOTE] Repeat the pull/tag/push pattern shown below for each dependency image listed in the
> [Runtime Fabric release notes](https://docs.mulesoft.com/release-notes/runtime-fabric/runtime-fabric-release-notes-2.x.x)
> for your version, not just `rtf-agent`.
1. Log in to your private container registry and run this command, replacing `docker-server` where
appropriate:
For the US Cloud:
```copy
# docker pull rtf-runtime-registry.kprod.msap.io/mulesoft/rtf-agent:v1.12.0
# docker tag rtf-runtime-registry.kprod.msap.io/mulesoft/rtf-agent:v1.12.0 /mulesoft/rtf-agent:v1.12.0
# docker push /mulesoft/rtf-agent:v1.12.0
```
For the EU Cloud:
```copy
# docker pull rtf-runtime-registry.kprod-eu.msap.io/mulesoft/rtf-agent:v1.12.0
# docker tag rtf-runtime-registry.kprod-eu.msap.io/mulesoft/rtf-agent:v1.12.0 /mulesoft/rtf-agent:v1.12.0
# docker push /mulesoft/rtf-agent:v1.12.0
```
For the Canada Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-58ktaz.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-58ktaz.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0 /mulesoft/rtf-agent:v1.12.0
# docker push /mulesoft/rtf-agent:v1.12.0
```
For the Japan Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-mchho0.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-mchho0.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0 /mulesoft/rtf-agent:v1.12.0
# docker push /mulesoft/rtf-agent:v1.12.0
```
For the India Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-y37hzm.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-y37hzm.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0 /mulesoft/rtf-agent:v1.12.0
# docker push /mulesoft/rtf-agent:v1.12.0
```
For the Australia Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-vwfla6.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-vwfla6.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0 /mulesoft/rtf-agent:v1.12.0
# docker push /mulesoft/rtf-agent:v1.12.0
```
For the United Kingdom Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-5pakla.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-5pakla.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-rtf-agent:v1.12.0 /mulesoft/rtf-agent:v1.12.0
# docker push /mulesoft/rtf-agent:v1.12.0
```
### Synchronize Mule Runtime Images
To synchronize the Mule runtime versions you plan to use for application deployments, refer to the
[Mule runtime patch updates for Runtime Fabric Release Notes](https://docs.mulesoft.com/release-notes/runtime-fabric/runtime-fabric-runtimes-release-notes)
for the image names and tags. The source registry depends on your control plane:
| Control Plane | Image Location Format |
|---|
US Cloud | rtf-runtime-registry.kprod.msap.io/mulesoft/<image>:<tag>
|
EU Cloud | rtf-runtime-registry.kprod-eu.msap.io/mulesoft/<image>:<tag>
|
Canada Cloud | rtf-runtime-registry-mulesoft-cp.sfdc-58ktaz.svc.sfdcfc.net/v2/sfci/mulesoft/ms-docker-image/<image>:<tag>
|
Japan Cloud | rtf-runtime-registry-mulesoft-cp.sfdc-mchho0.svc.sfdcfc.net/v2/sfci/mulesoft/ms-docker-image/<image>:<tag>
|
India Cloud | rtf-runtime-registry-mulesoft-cp.sfdc-y37hzm.svc.sfdcfc.net/v2/sfci/mulesoft/ms-docker-image/<image>:<tag>
|
Australia Cloud | rtf-runtime-registry-mulesoft-cp.sfdc-vwfla6.svc.sfdcfc.net/v2/sfci/mulesoft/ms-docker-image/<image>:<tag>
|
United Kingdom Cloud | rtf-runtime-registry-mulesoft-cp.sfdc-5pakla.svc.sfdcfc.net/v2/sfci/mulesoft/ms-docker-image/<image>:<tag>
|
> [!NOTE] For Canada Cloud, Japan Cloud, India Cloud, Australia Cloud, and United Kingdom Cloud
> deployments on OpenShift, download these images from the SFCI registry:
>
> - Mule runtime: `/sfci/mulesoft/ms-docker-image/ms-poseidon-runtime-:`
>
> - Anypoint Monitoring sidecar:
> `/sfci/mulesoft/ms-docker-image/ms-dias-anypoint-monitoring-sidecar-ubi:`
There is no automated tooling for Mule runtime images. For each runtime version you deploy, manually
pull the image from the MuleSoft source registry, retag it for your local registry, and push it,
using the same `docker pull`/`docker tag`/`docker push` pattern shown for the core software images.
Log in to both the MuleSoft source registry and your private container registry, then run these
commands for your control plane, replacing `` and `` with the values from the
[Mule runtime release notes](https://docs.mulesoft.com/release-notes/runtime-fabric/runtime-fabric-runtimes-release-notes)
and `` with your local registry server. These examples synchronize a single Mule
runtime image; repeat the pattern for each runtime version you deploy.
For the US Cloud:
```copy
# docker pull rtf-runtime-registry.kprod.msap.io/mulesoft/:
# docker tag rtf-runtime-registry.kprod.msap.io/mulesoft/: /mulesoft/:
# docker push /mulesoft/:
```
For the EU Cloud:
```copy
# docker pull rtf-runtime-registry.kprod-eu.msap.io/mulesoft/:
# docker tag rtf-runtime-registry.kprod-eu.msap.io/mulesoft/: /mulesoft/:
# docker push /mulesoft/:
```
For the Canada Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-58ktaz.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-:
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-58ktaz.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-: /mulesoft/:
# docker push /mulesoft/:
```
For the Japan Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-mchho0.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-:
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-mchho0.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-: /mulesoft/:
# docker push /mulesoft/:
```
For the India Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-y37hzm.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-:
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-y37hzm.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-: /mulesoft/:
# docker push /mulesoft/:
```
For the Australia Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-vwfla6.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-:
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-vwfla6.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-: /mulesoft/:
# docker push /mulesoft/:
```
For the United Kingdom Cloud:
```copy
# docker pull rtf-runtime-registry-mulesoft-cp.sfdc-5pakla.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-:
# docker tag rtf-runtime-registry-mulesoft-cp.sfdc-5pakla.svc.sfdcfc.net/sfci/mulesoft/ms-docker-image/ms-: /mulesoft/:
# docker push /mulesoft/:
```
> [!NOTE] For Canada Cloud, Japan Cloud, India Cloud, Australia Cloud, and United Kingdom Cloud, the
> source image in the SFCI registry uses an `ms-` prefix in the artifact name (for example,
> `ms-`). When you retag the image for your local registry, drop the `ms-` prefix so that
> Runtime Fabric can pull it using the expected name.
## Configure Your Local Registry for Runtime Fabric
To configure a local registry, you must gather and add the necessary credentials to synchronize the
registries.
### Before You Begin
Ensure that you’ve completed these steps before proceeding:
1. Set up, configure, and test your private Docker image registry.
2. [Synchronize](#synchronize-images-to-your-local-registry) all required Docker images to your
local registry. See the
[Runtime Fabric release notes](https://docs.mulesoft.com/release-notes/runtime-fabric/runtime-fabric-release-notes-2.x.x)
for the dependency versions specific to your Runtime Fabric version.
To install Runtime Fabric with your local registry, select your installation method:
- RTFCTL
- Helm
- OpenShift
1. [Download and install the `rtfctl` command line utility](https://docs.mulesoft.com/runtime-fabric/latest/install-self-managed#step-3-download-the-rtfctl-utility)
version 0.3.150 or higher.
2. Obtain `rtf-runtime-registry` credentials:
```copy
# rtfctl get registry-credentials
```
This command sets `RTF_IMAGE_REGISTRY_ENDPOINT`, `RTF_IMAGE_REGISTRY_USER`, and
`RTF_IMAGE_REGISTRY_PASSWORD` in the current shell environment.
3. Run these commands to verify that the Docker login to the `rtf-runtime-registry` succeeds:
```copy
# eval $(rtfctl get registry-credentials )
# docker login $RTF_IMAGE_REGISTRY_ENDPOINT --username $RTF_IMAGE_REGISTRY_USER --password $RTF_IMAGE_REGISTRY_PASSWORD
```
You should see a message that the login was successful.
4. [Synchronize your container images](#synch-rtf) to your local registry.
5. Repeat step 4 for all other dependency images (such as `resourceFetcher` and `clusterOps`) based
on the versions published in the Runtime Fabric release notes.
6. If you use authentication to access your registry, create the required secret in the rtf
namespace:
```copy
# kubectl create namespace rtf
# kubectl create secret docker-registry regcred --namespace=rtf --docker-server= --docker-username= --docker-email= --docker-password=
# kubectl label secret regcred rtf.mulesoft.com/synchronized=true -n rtf
```
7. Finalize the Runtime Fabric installation, and configure it to pull from your local registry:
```copy
# rtfctl install ‘’ --image-pull-registry
```
Alternatively, if you use authentication to access your registry, use this command:
```copy
# rtfctl install ‘’ --image-pull-registry --image-pull-secret
```
This command ensures that Runtime Fabric creates the necessary overrides for the local registry
URL and `image-pull-secret` values so Runtime Fabric continues to pull images from the local
registry even after an upgrade.
8. If you use authentication to access your registry, after you complete the installation, you must
recreate the pull secret in the RTF namespace and in any other namespaces where you will deploy
applications.
9. Refer to the [installation instructions](https://docs.mulesoft.com/runtime-fabric/latest/install-self-managed) for further steps, such as
inserting the Mule license key.
The `rtfctl uninstall` command deletes any pull secret you specify during installation.
When using `rtfctl` to perform a backup and restore procedure, the backups point to your local
registry URL and pull secret.
1. [Synchronize your container images](#synch-rtf) to your local registry.
2. [Create a Runtime Fabric using Runtime Manager](https://docs.mulesoft.com/runtime-fabric/latest/install-helm#create-a-runtime-fabric-using-runtime-manager).
3. In Runtime Manager, select the **Helm** path, and follow the on-screen instructions.
4. Obtain your private `docker-server`, `docker-username`, and `docker-password`.
5. Follow the **Helm** path instructions in Runtime Manager to create a secret with your docker
server name, username, and password by running this command in your Unix shell:
```copy
kubectl create secret docker-registry rtf-pull-secret --namespace --docker-server= --docker-username= --docker-email=
```
6. Follow the **Helm** path instructions to download the `values.yml` file so that you can modify
its values. For the private registry configuration, update `rtfRegistry` with your local docker
server, and update the `pullSecretName` parameter.
7. Continue with the **Helm** path instructions and install Runtime Fabric in your Kubernetes
cluster.
8. [Synchronize your container images](#synch-rtf) to your local registry.
9. [Create a Runtime Fabric using Runtime Manager](https://docs.mulesoft.com/runtime-fabric/latest/install-helm#create-a-runtime-fabric-using-runtime-manager).
10. In Runtime Manager, follow the on-screen instructions.
11. Obtain your private `docker-server`, `docker-username`, and `docker-password`.
12. Follow the instructions in Runtime Manager to create a secret with your docker server name,
username, and password by running this command in your Unix shell:
```copy
oc create secret docker-registry rtf-pull-secret --namespace --docker-server= --docker-username= --docker-email=
```
13. In the OpenShift form view, set the rtfRegistry values to configure the local registry.

14. Continue with the OpenShift installation.
## See Also
- [Configure values.yml Reference](https://docs.mulesoft.com/runtime-fabric/latest/install-helm#values-yml-reference)
---
product: Runtime Fabric
version: 2.10
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of the following Runtime Fabric
resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, the following RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, and GKE Using Helm](https://docs.mulesoft.com/runtime-fabric/2.10/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/2.10/install-openshift)
## Add New Namespaces
If you update the authorized-namespaces ConfigMap with a new namespace, you must restart the Runtime
Fabric agent deployment using the following command:
`kubectl rollout restart deployment/agent`
---
product: Runtime Fabric
version: 2.11
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of these Runtime Fabric resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, this RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, and GKE Using Helm](https://docs.mulesoft.com/runtime-fabric/2.11/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/2.11/install-openshift)
## Add New Namespaces
If you update the authorized-namespaces ConfigMap with a new namespace, you must restart the Runtime
Fabric agent deployment using this command:
`kubectl rollout restart deployment/agent`
---
product: Runtime Fabric
version: 2.5
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of the following Runtime Fabric
resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, the following RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, and GKE Using Helm](https://docs.mulesoft.com/runtime-fabric/2.5/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/2.5/install-openshift)
---
product: Runtime Fabric
version: 2.6
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of the following Runtime Fabric
resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, the following RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service-
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, and GKE Using Helm](https://docs.mulesoft.com/runtime-fabric/2.6/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/2.6/install-openshift)
## Add New Namespaces
If you update the authorized-namespaces ConfigMap with a new namespace, you must restart the Runtime
Fabric agent deployment using the following command:
`kubectl rollout restart deployment/agent`
---
product: Runtime Fabric
version: 2.7
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of the following Runtime Fabric
resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, the following RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service-
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, and GKE Using Helm](https://docs.mulesoft.com/runtime-fabric/2.7/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/2.7/install-openshift)
## Add New Namespaces
If you update the authorized-namespaces ConfigMap with a new namespace, you must restart the Runtime
Fabric agent deployment using the following command:
`kubectl rollout restart deployment/agent`
---
product: Runtime Fabric
version: 2.8
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of the following Runtime Fabric
resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, the following RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service-
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, and GKE Using Helm](https://docs.mulesoft.com/runtime-fabric/2.8/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/2.8/install-openshift)
## Add New Namespaces
If you update the authorized-namespaces ConfigMap with a new namespace, you must restart the Runtime
Fabric agent deployment using the following command:
`kubectl rollout restart deployment/agent`
---
product: Runtime Fabric
version: 2.9
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of the following Runtime Fabric
resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, the following RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, and GKE Using Helm](https://docs.mulesoft.com/runtime-fabric/2.9/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/2.9/install-openshift)
## Add New Namespaces
If you update the authorized-namespaces ConfigMap with a new namespace, you must restart the Runtime
Fabric agent deployment using the following command:
`kubectl rollout restart deployment/agent`
---
product: Runtime Fabric
version: 3.0
is-latest-version: true
---
# Configuring Authorized Namespaces in Runtime Fabric
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Authorized namespaces enable you to deploy Runtime Fabric alongside other services in a Kubernetes
cluster. By limiting permissions in your cluster, you can allow Runtime Fabric to access resources
only within namespaces assigned to Runtime Fabric.
With authorized namespaces, you have control of the creation of the following Runtime Fabric
resources:
- Namespaces
- RoleBindings
- ServiceAccounts
## How Authorized Namespaces Work
At install time, you provide a configuration file that consists of your namespaces, ServiceAccounts,
ClusterRoles/Roles, and RoleBindings.
Runtime Fabric installs the Runtime Fabric agent (rtf-agent) in the Runtime Fabric namespace, and it
has access to any additional namespaces that you provide in the authorized namespace list.
### Example
The following is an example of a Runtime Fabric agent ClusterRole that allows the agent to control
all cron jobs across all namespaces in a cluster.
```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rtf:agent
rules:
- apiGroups:
- batch
resources:
- cronjobs
verbs:
- create
- delete
- deletecollection
- get
- list
- patch
- update
- watch
```
Runtime Fabric creates the required RoleBinding objects which use the ClusterRole rtf:agent to allow
corresponding ServiceAccount to perform various actions (verbs) on cron jobs in two namespaces.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-1
namespace: namespace1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf:agent-2
namespace: namespace2
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rtf:agent
subjects:
- kind: ServiceAccount
name: rtf-agent-sa
namespace: default
```
To use clustered apps in authorized namespaces mode, the following RoleBinding must be created:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rtf-mule-clusterip-service
namespace:
subjects:
- kind: ServiceAccount
name: mule-clusterip-service
namespace:
roleRef:
kind: ClusterRole
name: rtf:mule-clusterip-service
apiGroup: rbac.authorization.k8s.io
```
## Configure Authorized Namespaces
You configure authorized namespaces when you install Runtime Fabric. Refer to the appropriate
installation instructions:
- [Installing Runtime Fabric on EKS, AKS, GKE, and OKE Using Helm](https://docs.mulesoft.com/runtime-fabric/latest/install-helm)
- [Install Runtime Fabric on Red Hat OpenShift](https://docs.mulesoft.com/runtime-fabric/latest/install-openshift)
## Add New Namespaces
If you update the authorized-namespaces ConfigMap with a new namespace, you must restart the Runtime
Fabric agent deployment using the following command:
`kubectl rollout restart deployment/agent`
---
product: Runtime Fabric
version: 2.10
---
# Configuring Horizontal Pod Autoscaling (HPA) for Runtime Fabric Deployments
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Runtime Fabric instances support horizontal scaling of Mule app deployments by spinning up
additional replicas or spinning down unneeded replicas in response to the CPU usage as a signal.
Mulesoft supports configuring Mule apps to be responsive to the CPU usage on Kubernetes (K8s)
runtime side. You can enable CPU based horizontal autoscaling per application on the Runtime Manager
application deployment page, which will result in automatic scale up or scale down of the deployment
replicas as needed.
In Kubernetes, a Horizontal Pod Autoscaler (HPA) automatically updates a workload resource, with the
aim of automatically scaling the workload to match demand. Horizontal scaling means that the
response to increased load is to deploy more pods.
> [!NOTE] HPA is supported only on Runtime Fabric agent version 2.6.22 and higher.
This feature applies only to select customers who are opted into the new pricing and packaging
model. For more details, see [Anypoint Platform Pricing](https://docs.mulesoft.com/general/pricing)
## Before You Begin
To configure autoscaling, the cluster administrator must enable the
[support for metrics APIs](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
on your managed K8s API server. Refer to your managed vendors documentation on how to install or
enable metrics API servers.
## Recommendations for Cluster Autoscaling
The following recommendations are for your infrastructure team to configure cluster autoscaling
### Configure Node Groups
Configure one node group per availability zone, with a minimum of one node in the node group. This
ensures that Kubernetes has information about all the availability zones in the cluster and avoids
an issue where zone topology unschedulable pod fails to scale-out topologySpread predicate check.
Make sure that nodes from all the node groups have the exact same labels except for the common
labels and specific cloud provider labels that are different across nodes or availability zones by
default.
Custom labels (which are not common labels and specific cloud provider labels) and are different
across nodes need to be excluded from cluster autoscaler when evaluating similar node groups.
For reference on managed EKS labels such as:
eks.amazonaws.com/nodegroup failure-domain.beta.kubernetes.io/region
failure-domain.beta.kubernetes.io/zone kubernetes.io/hostname topology.kubernetes.io/region
topology.kubernetes.io/zone
### Configure Cluster Autoscaler Flags
Start the cluster autoscaler with flag `--balance-similar-node-groups=true` and exclude the custom
labels previously mentioned in the node groups section with the flag `--balancing-ignore-label`.
These flags ensure that the cluster autoscaler balances the number of nodes across availability
zones.
For reference on managed EKS cluster, the following node labels are different across node groups:
k8s.amazonaws.com/eniConfigCustom vpc.amazonaws.com/eniConfig
Therefore, the cluster autoscaler starts with:
\- --balancing-ignore-label=k8s.amazonaws.com/eniConfigCustom
- --balancing-ignore-label=vpc.amazonaws.com/eniConfig
These configuration recommendations serve as a guideline. However, the actual configuration of the
cluster autoscaler (CA) depends on the cloud or infrastructure provider (vendor) you use and the
relevant topology considerations. You must configure the CA appropriately.
### Configure Overprovisioning Nodes
Configure at least one overprovisioning node per availability zone. Each overprovisioning nodes
holds a resource that can accommodate three Mule app nodes, which improves the performance when new
nodes are needed.
> [!NOTE] Refer to your managed vendor documentation for cluster autoscaling best practices and
> over-provisioning configuration details.
### Configure Topology Spread Constraints
Your Mule app deployments which are enabled with autoscaling, are required to customize the
kubernetes resources for the topology spread constraints in pod spec by updating the
**whenUnsatisfiable** field with `DoNotSchedule`. This configuration ensures that autoscaling
replicas are evenly spread across zones for high availability.
topologySpreadConstraints:
- maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: DoNotSchedule
labelSelector: matchLabels: rtf.mulesoft.com/id: "{{ .Values.id }}" rtf.mulesoft.com/generation:
"{{ .Values.resourceContentHash }}"
Using `whenUnsatisfiable: DoNotSchedule` guarantees that pods are evenly spread across availability
zones. To modify this setting, use the K8S templates CRD to adjust your deployments using HPA to
`whenUnsatisfiable: DoNotSchedule`. You can also use dedicated namespaces (environments) to separate
the autoscaling workloads from the classic ones. If you don’t make this change, you use
`whenUnsatisfiable: ScheduleAnyway`.
For further details on the configuration, refer to
[Customizing Mule App Kubernetes Resources](https://docs.mulesoft.com/runtime-fabric/2.10/customize-kubernetes-crd) documentation.
> [!WARNING] Customizing topology spread with `whenUnsatisfiable: DoNotSchedule`, without the
> recommended configurations for node groups per availability zone, cluster autoscaler, and
> overprovisionining, can lead to K8s failing to schedule the replicas with `Pending state` error.
### Understand CPU-based Autoscaling Policy
MuleSoft owns and applies the
[autoscaling](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
policy for your Mule application deployments.
The following example shows the CPU-based HPA policy used for Mule apps deployed on Runtime Fabric
instances. Note that you can choose `minReplicas` and `maxReplicas` values from the Runtime Manager
user interface.
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-app namespace:
app-namespace spec: behavior: scaleDown: policies: - periodSeconds: 15 type: Percent value: 100
selectPolicy: Max stabilizationWindowSeconds: 1800 scaleUp: policies: - periodSeconds: 180 type:
Percent value: 100 selectPolicy: Max stabilizationWindowSeconds: 0 maxReplicas:
metrics:
- resource: name: cpu target: averageUtilization: 70 type: Utilization type: Resource minReplicas:
scaleTargetRef: apiVersion: apps/v1 kind: Deployment name:
my-app
Some points to consider:
Scale up can occur at most every 180 seconds. Each period, up to 100% of the currently running
replicas may be added until the maximum configured replicas is reached. For scaling up there is no
stabilization window. When the metrics indicate that the target should be scaled up, the target is
scaled up immediately.
Scale down can occur at most every 15 seconds. Each Period, up to 100% of the currently running
replicas may be removed which means the scaling target can be scaled down to the minimum allowed
replicas. The number of replicas removed is based on the aggregated calculations over the past 1800
seconds of the stabilization window.
Min replicas
- The minimum number of replicas that would be guaranteed to run at any given point of time.
- Scale down policy would never remove replicas below this number.
Max replicas
- The maximum number of replicas that are capped, beyond which no more replicas can be added for
scale up. For details, see [limitations-self.adoc#limitations](https://docs.mulesoft.com/runtime-fabric/2.10/limitations-self#limitations).
- Scale up policy would never add replicas above this number.
Enabling HPA can result in customers incurring additional flow usage when your application scales
horizontally. To avoid overages from unpredicted scaling, configure the maximum configured replicas
judiciously to stay within purchased flow limits. Track your incurred flow usage through
[usage reports](https://docs.mulesoft.com/general/usage-reports).
HPA uses CPU utilization at average of 70%
HPA bases on
[resource requests](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/#how-does-a-horizontalpodautoscaler-work).
When target utilization value is set, the controller calculates the utilization value as a
percentage of the equivalent resource request on the containers in each pod.
If pod average use is greater than 70% in a period (periods for scale-up are set as 180 seconds),
HPA scales up. Refer to the Kubernetes documentation for additional details.
## Performance Considerations and Limitations
For a successful horizontal autoscaling of your Mule apps, review the following performance
considerations:
- In Runtime Fabric, the policy in use was benchmarked for Mule apps with CPU Reserved: 0.45vCpu and
Limit: 0.55vCpu, which corresponds to these settings:
resources:
limits:
cpu: 550m
requests:
cpu: 450m
- Mule apps that scale based on CPU usage are a good fit with CPU based HPA. For example:
- HTTP/HTTPS apps with async requests
- Reverse proxies
- Low latency + high throughput apps
- Dataweave transformations
- APIKit Routing
- API Gateways with policies
- Non-reentrant apps that do not have built in parallel processing such as batch jobs, scheduler
apps without re-entrancy and duplicate scheduling across apps and low throughput, high latency
apps with large requests may not be a good fit with CPU based HPA.
- Scale up and scale down performance can vary based on the replica size and the application type.
- The policy is optimized for replica sizes with <850m. Larger replica sizes might take longer to
scale. HPA is recommended only for smaller CPU applications with average usage of 0.2vCPU.
### Limitations
There are some limitations to consider:
- CPU is the only resource for horizontal autoscaling.
## Configure Horizontal Pod Autoscaling
To configure horizontal autoscaling for Mule apps deployed to Runtime Fabric, follow these steps:
1. Enable the
[support for metrics APIs](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
on your managed K8s API server.
2. From Anypoint Platform, select **Runtime Manager** > **Applications**.
3. Click **Deploy application**.
4. In the **Runtime** tab, check the **Enable Autoscaling** box.
5. Set the minimum and maximum **Replica Count** limits.
6. Click **Deploy Application**.

## Autoscaling Status and Logs
When an autoscaling event occurs and your Mule app with horizontal autoscaling scales up, you can
check the **Scaling** status in the Runtime Manager UI by clicking **View status** in your
application’s details window:

You can also track when autoscaling events occurred through **Audit logs** in Access Management.
Each time an application deployment scaled, there is an audit log published under the product
**Runtime Manager**, by the **Anypoint Staff** user. The log has **Action** set to `Scaling` with
the **Object** as the application ID.

The following is an example log payload:
```console
{"properties":{"organizationId":"my-orgID-abc","environmentId":"my-envID-xyz","response":{"message":{"message":"Application id:my-appID-123 scaled DOWN from 3 to 2 replicas.","logLevel":"INFO","context":{"logger":"Runtime Manager"},"timestamp":1700234556678}},"deploymentId":"my-appID-123","initialRequest":"/organizations/my-orgID-abc/environments/my-envID-xyz/deployments/my-appID-123/specs/my-specID-456"},"subaction":"Scaling"}
```
Additionally, you can track the scaled up replicas startup and the number of replicas of your Mule
apps by running the following `kubectl` command in your terminal:
```console
kubectl get events | grep HorizontalPodAutoscaler
```
Use the `kubectl get events` command in Kubernetes to retrieve events that occurred within the
cluster. The command provides information about various activities and changes happening in the
cluster, such as pod creations, deletions, and other important events.
The `grep` command filters the output of `kubectl get events` for lines that contain the string
`HorizontalPodAutoscaler`. The following example shows the command output that includes events
related to a `HorizontalPodAutoscaler` with information about successful rescaling operations
triggered by the HPA:
```console
# kubectl get events | grep HorizontalPodAutoscaler
5m20s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target
5m5s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 8; reason: cpu resource utilization (percentage of request) above target
4m50s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 10; reason:
```
---
product: Runtime Fabric
version: 2.11
---
# Configuring Horizontal Pod Autoscaling (HPA) for Runtime Fabric Deployments
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Runtime Fabric instances support horizontal scaling of Mule app deployments by spinning up
additional replicas or spinning down unneeded replicas in response to the CPU usage as a signal.
Mulesoft supports configuring Mule apps to be responsive to the CPU usage on Kubernetes (K8s)
runtime side. You can enable CPU based horizontal autoscaling per application on the Runtime Manager
application deployment page, which will result in automatic scale up or scale down of the deployment
replicas as needed.
In Kubernetes, a Horizontal Pod Autoscaler (HPA) automatically updates a workload resource, with the
aim of automatically scaling the workload to match demand. Horizontal scaling means that the
response to increased load is to deploy more pods.
> [!NOTE] HPA is supported only on Runtime Fabric agent version 2.6.22 and higher.
This feature applies only to select customers who are opted into the new pricing and packaging
model. For more details, see [Anypoint Platform Pricing](https://docs.mulesoft.com/general/pricing)
## Before You Begin
To configure autoscaling, the cluster administrator must enable the
[support for metrics APIs](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
on your managed K8s API server. Refer to your managed vendors documentation on how to install or
enable metrics API servers.
## Recommendations for Cluster Autoscaling
The following recommendations are for your infrastructure team to configure cluster autoscaling
### Configure Node Groups
Configure one node group per availability zone, with a minimum of one node in the node group. This
ensures that Kubernetes has information about all the availability zones in the cluster and avoids
an issue where zone topology unschedulable pod fails to scale-out topologySpread predicate check.
Make sure that nodes from all the node groups have the exact same labels except for the common
labels and specific cloud provider labels that are different across nodes or availability zones by
default.
Custom labels (which are not common labels and specific cloud provider labels) and are different
across nodes need to be excluded from cluster autoscaler when evaluating similar node groups.
For reference on managed EKS labels such as:
eks.amazonaws.com/nodegroup failure-domain.beta.kubernetes.io/region
failure-domain.beta.kubernetes.io/zone kubernetes.io/hostname topology.kubernetes.io/region
topology.kubernetes.io/zone
### Configure Cluster Autoscaler Flags
Start the cluster autoscaler with flag `--balance-similar-node-groups=true` and exclude the custom
labels previously mentioned in the node groups section with the flag `--balancing-ignore-label`.
These flags ensure that the cluster autoscaler balances the number of nodes across availability
zones.
For reference on managed EKS cluster, these node labels are different across node groups:
k8s.amazonaws.com/eniConfigCustom vpc.amazonaws.com/eniConfig
Therefore, the cluster autoscaler starts with:
\- --balancing-ignore-label=k8s.amazonaws.com/eniConfigCustom
- --balancing-ignore-label=vpc.amazonaws.com/eniConfig
These configuration recommendations serve as a guideline. However, the actual configuration of the
cluster autoscaler (CA) depends on the cloud or infrastructure provider (vendor) you use and the
relevant topology considerations. You must configure the CA appropriately.
### Configure Overprovisioning Nodes
Configure at least one overprovisioning node per availability zone. Each overprovisioning nodes
holds a resource that can accommodate three Mule app nodes, which improves the performance when new
nodes are needed.
> [!NOTE] Refer to your managed vendor documentation for cluster autoscaling best practices and
> over-provisioning configuration details.
### Configure Topology Spread Constraints
Your Mule app deployments which are enabled with autoscaling, are required to customize the
kubernetes resources for the topology spread constraints in pod spec by updating the
**whenUnsatisfiable** field with `DoNotSchedule`. This configuration ensures that autoscaling
replicas are evenly spread across zones for high availability.
topologySpreadConstraints:
- maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: DoNotSchedule
labelSelector: matchLabels: rtf.mulesoft.com/id: "{{ .Values.id }}" rtf.mulesoft.com/generation:
"{{ .Values.resourceContentHash }}"
Using `whenUnsatisfiable: DoNotSchedule` guarantees that pods are evenly spread across availability
zones. To modify this setting, use the K8S templates CRD to adjust your deployments using HPA to
`whenUnsatisfiable: DoNotSchedule`. You can also use dedicated namespaces (environments) to separate
the autoscaling workloads from the classic ones. If you don’t make this change, you use
`whenUnsatisfiable: ScheduleAnyway`.
For further details on the configuration, refer to
[Customizing Mule App Kubernetes Resources](https://docs.mulesoft.com/runtime-fabric/2.11/customize-kubernetes-crd) documentation.
> [!WARNING] Customizing topology spread with `whenUnsatisfiable: DoNotSchedule`, without the
> recommended configurations for node groups per availability zone, cluster autoscaler, and
> overprovisionining, can lead to K8s failing to schedule the replicas with `Pending state` error.
### Understand CPU-based Autoscaling Policy
MuleSoft owns and applies the
[autoscaling](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
policy for your Mule application deployments.
The following example shows the CPU-based HPA policy used for Mule apps deployed on Runtime Fabric
instances. Note that you can choose `minReplicas` and `maxReplicas` values from the Runtime Manager
user interface.
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-app namespace:
app-namespace spec: behavior: scaleDown: policies: - periodSeconds: 15 type: Percent value: 100
selectPolicy: Max stabilizationWindowSeconds: 1800 scaleUp: policies: - periodSeconds: 180 type:
Percent value: 100 selectPolicy: Max stabilizationWindowSeconds: 0 maxReplicas:
metrics:
- resource: name: cpu target: averageUtilization: 70 type: Utilization type: Resource minReplicas:
scaleTargetRef: apiVersion: apps/v1 kind: Deployment name:
my-app
Some points to consider:
Scale up can occur at most every 180 seconds. Each period, up to 100% of the currently running
replicas may be added until the maximum configured replicas is reached. For scaling up there is no
stabilization window. When the metrics indicate that the target should be scaled up, the target is
scaled up immediately.
Scale down can occur at most every 15 seconds. Each Period, up to 100% of the currently running
replicas may be removed which means the scaling target can be scaled down to the minimum allowed
replicas. The number of replicas removed is based on the aggregated calculations over the past 1800
seconds of the stabilization window.
Min replicas
- The minimum number of replicas that would be guaranteed to run at any given point of time.
- Scale down policy would never remove replicas below this number.
Max replicas
- The maximum number of replicas that are capped, beyond which no more replicas can be added for
scale up. For details, see [limitations-self.adoc#limitations](https://docs.mulesoft.com/runtime-fabric/2.11/limitations-self#limitations).
- Scale up policy would never add replicas above this number.
Enabling HPA can result in customers incurring additional flow usage when your application scales
horizontally. To avoid overages from unpredicted scaling, configure the maximum configured replicas
judiciously to stay within purchased flow limits. Track your incurred flow usage through
[usage reports](https://docs.mulesoft.com/general/usage-reports).
HPA uses CPU utilization at average of 70%
HPA bases on
[resource requests](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/#how-does-a-horizontalpodautoscaler-work).
When target utilization value is set, the controller calculates the utilization value as a
percentage of the equivalent resource request on the containers in each pod.
If pod average use is greater than 70% in a period (periods for scale-up are set as 180 seconds),
HPA scales up. Refer to the Kubernetes documentation for additional details.
## Performance Considerations and Limitations
For a successful horizontal autoscaling of your Mule apps, review these performance considerations:
- In Runtime Fabric, the policy in use was benchmarked for Mule apps with CPU Reserved: 0.45vCpu and
Limit: 0.55vCpu, which corresponds to these settings:
resources:
limits:
cpu: 550m
requests:
cpu: 450m
- Mule apps that scale based on CPU usage are a good fit with CPU based HPA. For example:
- HTTP/HTTPS apps with async requests
- Reverse proxies
- Low latency + high throughput apps
- Dataweave transformations
- APIKit Routing
- API Gateways with policies
- Non-reentrant apps that do not have built in parallel processing such as batch jobs, scheduler
apps without re-entrancy and duplicate scheduling across apps and low throughput, high latency
apps with large requests may not be a good fit with CPU based HPA.
- Scale up and scale down performance can vary based on the replica size and the application type.
- The policy is optimized for replica sizes with <850m. Larger replica sizes might take longer to
scale. HPA is recommended only for smaller CPU applications with average usage of 0.2vCPU.
### Limitations
There are some limitations to consider:
- CPU is the only resource for horizontal autoscaling.
## Configure Horizontal Pod Autoscaling
To configure horizontal autoscaling for Mule apps deployed to Runtime Fabric, follow these steps:
1. Enable the
[support for metrics APIs](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
on your managed K8s API server.
2. From Anypoint Platform, select **Runtime Manager** > **Applications**.
3. Click **Deploy application**.
4. In the **Runtime** tab, check the **Enable Autoscaling** box.
5. Set the minimum and maximum **Replica Count** limits.
6. Click **Deploy Application**.

## Autoscaling Status and Logs
When an autoscaling event occurs and your Mule app with horizontal autoscaling scales up, you can
check the **Scaling** status in the Runtime Manager UI by clicking **View status** in your
application’s details window:

You can also track when autoscaling events occurred through **Audit logs** in Access Management.
Each time an application deployment scaled, there is an audit log published under the product
**Runtime Manager**, by the **Anypoint Staff** user. The log has **Action** set to `Scaling` with
the **Object** as the application ID.

The following is an example log payload:
```console
{"properties":{"organizationId":"my-orgID-abc","environmentId":"my-envID-xyz","response":{"message":{"message":"Application id:my-appID-123 scaled DOWN from 3 to 2 replicas.","logLevel":"INFO","context":{"logger":"Runtime Manager"},"timestamp":1700234556678}},"deploymentId":"my-appID-123","initialRequest":"/organizations/my-orgID-abc/environments/my-envID-xyz/deployments/my-appID-123/specs/my-specID-456"},"subaction":"Scaling"}
```
Additionally, you can track the scaled up replicas startup and the number of replicas of your Mule
apps by running this `kubectl` command in your terminal:
```console
kubectl get events | grep HorizontalPodAutoscaler
```
Use the `kubectl get events` command in Kubernetes to retrieve events that occurred within the
cluster. The command provides information about various activities and changes happening in the
cluster, such as pod creations, deletions, and other important events.
The `grep` command filters the output of `kubectl get events` for lines that contain the string
`HorizontalPodAutoscaler`. The following example shows the command output that includes events
related to a `HorizontalPodAutoscaler` with information about successful rescaling operations
triggered by the HPA:
```console
# kubectl get events | grep HorizontalPodAutoscaler
5m20s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target
5m5s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 8; reason: cpu resource utilization (percentage of request) above target
4m50s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 10; reason:
```
---
product: Runtime Fabric
version: 2.6
---
# Configuring Horizontal Pod Autoscaling (HPA) for Runtime Fabric Deployments
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Runtime Fabric instances support horizontal scaling of Mule app deployments by spinning up
additional replicas or spinning down unneeded replicas in response to the CPU usage as a signal.
Mulesoft supports configuring Mule apps to be responsive to the CPU usage on Kubernetes (K8s)
runtime side. You can enable CPU based horizontal autoscaling per application on the Runtime Manager
application deployment page, which will result in automatic scale up or scale down of the deployment
replicas as needed.
In Kubernetes, a Horizontal Pod Autoscaler (HPA) automatically updates a workload resource, with the
aim of automatically scaling the workload to match demand. Horizontal scaling means that the
response to increased load is to deploy more pods.
> [!NOTE] HPA is supported only on Runtime Fabric agent version 2.6.22 and higher.
This feature applies only to select customers who are opted into the new pricing and packaging
model. For more details, see [Anypoint Platform Pricing](https://docs.mulesoft.com/general/pricing)
## Before You Begin
To configure autoscaling, the cluster administrator must enable the
[support for metrics APIs](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
on your managed K8s API server. Refer to your managed vendors documentation on how to install or
enable metrics API servers.
## Recommendations for Cluster Autoscaling
The following recommendations are for your infrastructure team to configure cluster autoscaling
### Configure Node Groups
Configure one node group per availability zone, with a minimum of one node in the node group. This
ensures that Kubernetes has information about all the availability zones in the cluster and avoids
an issue where zone topology unschedulable pod fails to scale-out topologySpread predicate check.
Make sure that nodes from all the node groups have the exact same labels except for the common
labels and specific cloud provider labels that are different across nodes or availability zones by
default.
Custom labels (which are not common labels and specific cloud provider labels) and are different
across nodes need to be excluded from cluster autoscaler when evaluating similar node groups.
For reference on managed EKS labels such as:
eks.amazonaws.com/nodegroup failure-domain.beta.kubernetes.io/region
failure-domain.beta.kubernetes.io/zone kubernetes.io/hostname topology.kubernetes.io/region
topology.kubernetes.io/zone
### Configure Cluster Autoscaler Flags
Start the cluster autoscaler with flag `--balance-similar-node-groups=true` and exclude the custom
labels previously mentioned in the node groups section with the flag `--balancing-ignore-label`.
These flags ensure that the cluster autoscaler balances the number of nodes across availability
zones.
For reference on managed EKS cluster, the following node labels are different across node groups:
k8s.amazonaws.com/eniConfigCustom vpc.amazonaws.com/eniConfig
Therefore, the cluster autoscaler starts with:
\- --balancing-ignore-label=k8s.amazonaws.com/eniConfigCustom
- --balancing-ignore-label=vpc.amazonaws.com/eniConfig
These configuration recommendations serve as a guideline. However, the actual configuration of the
cluster autoscaler (CA) depends on the cloud or infrastructure provider (vendor) you use and the
relevant topology considerations. You must configure the CA appropriately.
### Configure Overprovisioning Nodes
Configure at least one overprovisioning node per availability zone. Each overprovisioning nodes
holds a resource that can accommodate three Mule app nodes, which improves the performance when new
nodes are needed.
> [!NOTE] Refer to your managed vendor documentation for cluster autoscaling best practices and
> over-provisioning configuration details.
### Configure Topology Spread Constraints
Your Mule app deployments which are enabled with autoscaling, are required to customize the
kubernetes resources for the topology spread constraints in pod spec by updating the
**whenUnsatisfiable** field with `DoNotSchedule`. This configuration ensures that autoscaling
replicas are evenly spread across zones for high availability.
topologySpreadConstraints:
- maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: DoNotSchedule
labelSelector: matchLabels: rtf.mulesoft.com/id: "{{ .Values.id }}"
Using `whenUnsatisfiable: DoNotSchedule` guarantees that pods are evenly spread across availability
zones. To modify this setting, use the K8S templates CRD to adjust your deployments using HPA to
`whenUnsatisfiable: DoNotSchedule`. You can also use dedicated namespaces (environments) to separate
the autoscaling workloads from the classic ones. If you don’t make this change, you use
`whenUnsatisfiable: ScheduleAnyway`.
For further details on the configuration, refer to
[Customizing Mule App Kubernetes Resources](https://docs.mulesoft.com/runtime-fabric/2.6/customize-kubernetes-crd) documentation.
> [!WARNING] Customizing topology spread with `whenUnsatisfiable: DoNotSchedule`, without the
> recommended configurations for node groups per availability zone, cluster autoscaler, and
> overprovisionining, can lead to K8s failing to schedule the replicas with `Pending state` error.
### Understand CPU-based Autoscaling Policy
MuleSoft owns and applies the
[autoscaling](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
policy for your Mule application deployments.
The following example shows the CPU-based HPA policy used for Mule apps deployed on Runtime Fabric
instances. Note that you can choose `minReplicas` and `maxReplicas` values from the Runtime Manager
user interface.
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-app namespace:
app-namespace spec: behavior: scaleDown: policies: - periodSeconds: 15 type: Percent value: 100
selectPolicy: Max stabilizationWindowSeconds: 1800 scaleUp: policies: - periodSeconds: 180 type:
Percent value: 100 selectPolicy: Max stabilizationWindowSeconds: 0 maxReplicas:
metrics:
- resource: name: cpu target: averageUtilization: 70 type: Utilization type: Resource minReplicas:
scaleTargetRef: apiVersion: apps/v1 kind: Deployment name:
my-app
Some points to consider:
Scale up can occur at most every 180 seconds. Each period, up to 100% of the currently running
replicas may be added until the maximum configured replicas is reached. For scaling up there is no
stabilization window. When the metrics indicate that the target should be scaled up, the target is
scaled up immediately.
Scale down can occur at most every 15 seconds. Each Period, up to 100% of the currently running
replicas may be removed which means the scaling target can be scaled down to the minimum allowed
replicas. The number of replicas removed is based on the aggregated calculations over the past 1800
seconds of the stabilization window.
Min replicas
- The minimum number of replicas that would be guaranteed to run at any given point of time.
- Scale down policy would never remove replicas below this number.
Max replicas
- The maximum number of replicas that are capped, beyond which no more replicas can be added for
scale up.
- Scale up policy would never add replicas above this number.
Enabling HPA can result in customers incurring additional flow usage when your application scales
horizontally. To avoid overages from unpredicted scaling, configure the maximum configured replicas
judiciously to stay within purchased flow limits. Track your incurred flow usage through
[usage reports](https://docs.mulesoft.com/general/usage-reports).
HPA uses CPU utilization at average of 70%
HPA bases on
[resource requests](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/#how-does-a-horizontalpodautoscaler-work).
When target utilization value is set, the controller calculates the utilization value as a
percentage of the equivalent resource request on the containers in each pod.
If pod average use is greater than 70% in a period (periods for scale-up are set as 180 seconds),
HPA scales up. Refer to the Kubernetes documentation for additional details.
## Performance Considerations and Limitations
For a successful horizontal autoscaling of your Mule apps, review the following performance
considerations:
- In Runtime Fabric, the policy in use was benchmarked for Mule apps with CPU Reserved: 0.45vCpu and
Limit: 0.55vCpu, which corresponds to these settings:
resources:
limits:
cpu: 550m
requests:
cpu: 450m
- Mule apps that scale based on CPU usage are a good fit with CPU based HPA. For example:
- HTTP/HTTPS apps with async requests
- Reverse proxies
- Low latency + high throughput apps
- Dataweave transformations
- APIKit Routing
- API Gateways with policies
- Non-reentrant apps that do not have built in parallel processing such as batch jobs, scheduler
apps without re-entrancy and duplicate scheduling across apps and low throughput, high latency
apps with large requests may not be a good fit with CPU based HPA.
- Scale up and scale down performance can vary based on the replica size and the application type.
- The policy is optimized for replica sizes with <850m. Larger replica sizes might take longer to
scale. HPA is recommended only for smaller CPU applications with average usage of 0.2vCPU.
### Limitations
There are some limitations to consider:
- CPU is the only resource for horizontal autoscaling.
## Configure Horizontal Pod Autoscaling
To configure horizontal autoscaling for Mule apps deployed to Runtime Fabric, follow these steps:
1. Enable the
[support for metrics APIs](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
on your managed K8s API server.
2. From Anypoint Platform, select **Runtime Manager** > **Applications**.
3. Click **Deploy application**.
4. In the **Runtime** tab, check the **Enable Autoscaling** box.
5. Set the minimum and maximum **Replica Count** limits.
6. Click **Deploy Application**.

## Autoscaling Status and Logs
When an autoscaling event occurs and your Mule app with horizontal autoscaling scales up, you can
check the **Scaling** status in the Runtime Manager UI by clicking **View status** in your
application’s details window:

You can also track when autoscaling events occurred through **Audit logs** in Access Management.
Each time an application deployment scaled, there is an audit log published under the product
**Runtime Manager**, by the **Anypoint Staff** user. The log has **Action** set to `Scaling` with
the **Object** as the application ID.

The following is an example log payload:
```console
{"properties":{"organizationId":"my-orgID-abc","environmentId":"my-envID-xyz","response":{"message":{"message":"Application id:my-appID-123 scaled DOWN from 3 to 2 replicas.","logLevel":"INFO","context":{"logger":"Runtime Manager"},"timestamp":1700234556678}},"deploymentId":"my-appID-123","initialRequest":"/organizations/my-orgID-abc/environments/my-envID-xyz/deployments/my-appID-123/specs/my-specID-456"},"subaction":"Scaling"}
```
Additionally, you can track the scaled up replicas startup and the number of replicas of your Mule
apps by running the following `kubectl` command in your terminal:
```console
kubectl get events | grep HorizontalPodAutoscaler
```
Use the `kubectl get events` command in Kubernetes to retrieve events that occurred within the
cluster. The command provides information about various activities and changes happening in the
cluster, such as pod creations, deletions, and other important events.
The `grep` command filters the output of `kubectl get events` for lines that contain the string
`HorizontalPodAutoscaler`. The following example shows the command output that includes events
related to a `HorizontalPodAutoscaler` with information about successful rescaling operations
triggered by the HPA:
```console
# kubectl get events | grep HorizontalPodAutoscaler
5m20s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target
5m5s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 8; reason: cpu resource utilization (percentage of request) above target
4m50s Normal SuccessfulRescale HorizontalPodAutoscaler New size: 10; reason:
```
---
product: Runtime Fabric
version: 2.7
---
# Configuring Horizontal Pod Autoscaling (HPA) for Runtime Fabric Deployments
> For the full documentation index, see: https://docs.mulesoft.com/llms.txt
Runtime Fabric instances support horizontal scaling of Mule app deployments by spinning up
additional replicas or spinning down unneeded replicas in response to the CPU usage as a signal.
Mulesoft supports configuring Mule apps to be responsive to the CPU usage on Kubernetes (K8s)
runtime side. You can enable CPU based horizontal autoscaling per application on the Runtime Manager
application deployment page, which will result in automatic scale up or scale down of the deployment
replicas as needed.
In Kubernetes, a Horizontal Pod Autoscaler (HPA) automatically updates a workload resource, with the
aim of automatically scaling the workload to match demand. Horizontal scaling means that the
response to increased load is to deploy more pods.
> [!NOTE] HPA is supported only on Runtime Fabric agent version 2.6.22 and higher.
This feature applies only to select customers who are opted into the new pricing and packaging
model. For more details, see [Anypoint Platform Pricing](https://docs.mulesoft.com/general/pricing)
## Before You Begin
To configure autoscaling, the cluster administrator must enable the
[support for metrics APIs](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
on your managed K8s API server. Refer to your managed vendors documentation on how to install or
enable metrics API servers.
## Recommendations for Cluster Autoscaling
The following recommendations are for your infrastructure team to configure cluster autoscaling
### Configure Node Groups
Configure one node group per availability zone, with a minimum of one node in the node group. This
ensures that Kubernetes has information about all the availability zones in the cluster and avoids
an issue where zone topology unschedulable pod fails to scale-out topologySpread predicate check.
Make sure that nodes from all the node groups have the exact same labels except for the common
labels and specific cloud provider labels that are different across nodes or availability zones by
default.
Custom labels (which are not common labels and specific cloud provider labels) and are different
across nodes need to be excluded from cluster autoscaler when evaluating similar node groups.
For reference on managed EKS labels such as:
eks.amazonaws.com/nodegroup failure-domain.beta.kubernetes.io/region
failure-domain.beta.kubernetes.io/zone kubernetes.io/hostname topology.kubernetes.io/region
topology.kubernetes.io/zone
### Configure Cluster Autoscaler Flags
Start the cluster autoscaler with flag `--balance-similar-node-groups=true` and exclude the custom
labels previously mentioned in the node groups section with the flag `--balancing-ignore-label`.
These flags ensure that the cluster autoscaler balances the number of nodes across availability
zones.
For reference on managed EKS cluster, the following node labels are different across node groups:
k8s.amazonaws.com/eniConfigCustom vpc.amazonaws.com/eniConfig
Therefore, the cluster autoscaler starts with:
\- --balancing-ignore-label=k8s.amazonaws.com/eniConfigCustom
- --balancing-ignore-label=vpc.amazonaws.com/eniConfig
These configuration recommendations serve as a guideline. However, the actual configuration of the
cluster autoscaler (CA) depends on the cloud or infrastructure provider (vendor) you use and the
relevant topology considerations. You must configure the CA appropriately.
### Configure Overprovisioning Nodes
Configure at least one overprovisioning node per availability zone. Each overprovisioning nodes
holds a resource that can accommodate three Mule app nodes, which improves the performance when new
nodes are needed.
> [!NOTE] Refer to your managed vendor documentation for cluster autoscaling best practices and
> over-provisioning configuration details.
### Configure Topology Spread Constraints
Your Mule app deployments which are enabled with autoscaling, are required to customize the
kubernetes resources for the topology spread constraints in pod spec by updating the
**whenUnsatisfiable** field with `DoNotSchedule`. This configuration ensures that autoscaling
replicas are evenly spread across zones for high availability.
topologySpreadConstraints:
- maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: DoNotSchedule
labelSelector: matchLabels: rtf.mulesoft.com/id: "{{ .Values.id }}"
Using `whenUnsatisfiable: DoNotSchedule` guarantees that pods are evenly spread across availability
zones. To modify this setting, use the K8S templates CRD to adjust your deployments using HPA to
`whenUnsatisfiable: DoNotSchedule`. You can also use dedicated namespaces (environments) to separate
the autoscaling workloads from the classic ones. If you don’t make this change, you use
`whenUnsatisfiable: ScheduleAnyway`.
For further details on the configuration, refer to
[Customizing Mule App Kubernetes Resources](https://docs.mulesoft.com/runtime-fabric/2.7/customize-kubernetes-crd) documentation.
> [!WARNING] Customizing topology spread with `whenUnsatisfiable: DoNotSchedule`, without the
> recommended configurations for node groups per availability zone, cluster autoscaler, and
> overprovisionining, can lead to K8s failing to schedule the replicas with `Pending state` error.
### Understand CPU-based Autoscaling Policy
MuleSoft owns and applies the
[autoscaling](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/)
policy for your Mule application deployments.
The following example shows the CPU-based HPA policy used for Mule apps deployed on Runtime Fabric
instances. Note that you can choose `minReplicas` and `maxReplicas` values from the Runtime Manager
user interface.
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-app namespace:
app-namespace spec: behavior: scaleDown: policies: - periodSeconds: 15 type: Percent value: 100
selectPolicy: Max stabilizationWindowSeconds: 1800 scaleUp: policies: - periodSeconds: 180 type:
Percent value: 100 selectPolicy: Max stabilizationWindowSeconds: 0 maxReplicas: