From 0d9da1a4f6a1ddb434d86844365fe308ad085727 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 12 Feb 2026 09:53:18 +0530 Subject: [PATCH 1/2] extensions: sync and download For: https://github.com/apache/cloudstack/pull/11814 Signed-off-by: Abhishek Kumar --- source/adminguide/extensions.rst | 2 + .../adminguide/extensions/sync_download.rst | 257 ++++++++++++++++++ 2 files changed, 259 insertions(+) create mode 100644 source/adminguide/extensions/sync_download.rst diff --git a/source/adminguide/extensions.rst b/source/adminguide/extensions.rst index 2c6b7308e0..e62ecca2c8 100644 --- a/source/adminguide/extensions.rst +++ b/source/adminguide/extensions.rst @@ -94,6 +94,8 @@ CloudStack provides built-in Orchestrator Extensions for Proxmox, Hyper-V, and M .. include:: extensions/inbuilt_extensions.rst +.. include:: extensions/sync_download.rst + .. include:: extensions/limitations.rst .. include:: extensions/troubleshooting.rst diff --git a/source/adminguide/extensions/sync_download.rst b/source/adminguide/extensions/sync_download.rst new file mode 100644 index 0000000000..20a8254bf9 --- /dev/null +++ b/source/adminguide/extensions/sync_download.rst @@ -0,0 +1,257 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information# + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + + +Synchronization and Download +============================ + +In clustered Apache CloudStack management server deployments, extension files +must remain consistent across all management nodes. This release introduces +built-in support for: + +* Synchronizing extension files across management servers +* Downloading extension files as an archive +* Secure file sharing via Secondary Storage (SSVM) when direct access to a + management server is not possible + +These capabilities are available via new APIs and UI actions. + +Share Endpoint Configuration +---------------------------- + +Extension download and synchronization workflows rely on the management +server's **share endpoint**, which enables controlled file sharing. + +The share endpoint is configured in ``server.properties`` and allows +administrators to enable or disable file sharing, define the base directory, +configure cache behavior, and secure download links. + +Share Endpoint Properties +~~~~~~~~~~~~~~~~~~~~~~~~~ + ++----------------------+----------------------------------------------+---------------------------------------------+ +| Property | Default Value | Description | ++======================+==============================================+=============================================+ +| share.enabled | true | Enables or disables the file sharing | +| | | feature. Must be ``true`` for extension | +| | | downloads to function. | ++----------------------+----------------------------------------------+---------------------------------------------+ +| share.base.dir | /share | Base directory from which files can be | +| | | shared. If not explicitly set, the default | +| | | directory under the CloudStack user home | +| | | is used. | ++----------------------+----------------------------------------------+---------------------------------------------+ +| share.cache.control | public,max-age=86400,immutable | Cache-Control header value applied to | +| | | shared files. Controls browser/client | +| | | caching behavior. | ++----------------------+----------------------------------------------+---------------------------------------------+ +| share.secret | change-me | Secret key used to generate HMAC-signed | +| | | download links. It is strongly recommended | +| | | to change this value in production. If not | +| | | set, links will not be signed. | ++----------------------+----------------------------------------------+---------------------------------------------+ + +Notes: + +* ``share.enabled`` must be ``true`` for extension downloads to function. +* If ``share.secret`` is configured, generated links are signed using HMAC. +* It is strongly recommended to replace the default ``share.secret`` value + in production environments. +* If ``share.secret`` is not configured, download links will not be signed. + +Extension Synchronization +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In a multi-node management server cluster, extension files are stored locally +on each management server. If an extension is installed or modified on one +node, it must be synchronized to peers. + +syncExtension API +~~~~~~~~~~~~~~~~~ + +The ``syncExtension`` API triggers synchronization of a selected extension +across all management servers in the cluster. + +How It Works +~~~~~~~~~~~~ + +1. The source management server: + + * Calculates checksums for all files in the extension directory. + * Packages the extension directory (or selected files) into a ``.tgz`` archive. + +2. The server sends a ``DownloadAndSyncExtensionFilesCommand`` to peer + management servers. + +3. Each peer management server: + + * Downloads the archive. + * Stages it temporarily. + * Extracts the files into the extension directory. + * Validates checksums. + * Cleans up temporary files. + +Checksum Validation +~~~~~~~~~~~~~~~~~~~ + +Checksums are calculated for all files in the extension directory and compared +during synchronization. This ensures: + +* File integrity +* Detection of missing or modified files +* Cluster-wide consistency + +UI Support +~~~~~~~~~~ + +In the CloudStack UI: + +1. Navigate to **Extensions** +2. Select an extension +3. Click **Sync Extension** + +This triggers synchronization across all management servers. + +Downloading Extension Files +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You can download an extension’s files as an archive for backup, inspection, +or migration. + +downloadExtension API +~~~~~~~~~~~~~~~~~~~~~ + +The ``downloadExtension`` API allows administrators to download the complete +extension directory as a ``.zip`` archive. + +How It Works +~~~~~~~~~~~~ + +1. The management server creates a ``.zip`` archive of the extension directory. +2. A secure share URL is generated. +3. The administrator downloads the archive using the signed URL. + +Secure Share URL +~~~~~~~~~~~~~~~~ + +Download URLs are: + +* Signed using HMAC +* Time-bound (expire automatically) +* Cleaned up after expiry + +The validity interval is controlled via the global configuration: + +.. code-block:: text + + extension.share.link.validity.interval + +This defines how long (in seconds) the signed URL remains valid before +automatic cleanup. + +Download via Secondary Storage (SSVM) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In some environments, a management server may not be directly accessible from +the administrator’s network. + +To support this scenario: + +* The archive can be staged on Secondary Storage. +* The signed URL points to the SSVM endpoint. +* The file is served via Secondary Storage instead of directly from the + management server. + +Global Configuration +~~~~~~~~~~~~~~~~~~~~ + +To force downloads via Secondary Storage: + +.. code-block:: text + + extension.share.download.use.secondary.storage = true + +When set to ``true``, extension downloads are always served via Secondary +Storage (SSVM). + +When set to ``false`` (default), downloads are served directly from the +selected management server. + +Global Configuration Parameters +-------------------------------- + ++-----------------------------------------------+--------------------------------------------------------------+ +| Name | Description | ++===============================================+==============================================================+ +| extension.share.download.use.secondary.storage| If ``true``, forces extension downloads via Secondary | +| | Storage (SSVM). | ++-----------------------------------------------+--------------------------------------------------------------+ +| extension.share.link.validity.interval | Validity duration (in seconds) of the signed download URL. | ++-----------------------------------------------+--------------------------------------------------------------+ + +Events and Logging +------------------ + +The following improvements are included: + +* Dedicated events for sync and download operations +* Improved logging for archive creation, transfer, extraction, and cleanup +* Automatic cleanup of temporary share files after expiry + +These changes improve traceability and operational visibility. + +Archive Formats +--------------- + ++---------------+----------+ +| Operation | Format | ++===============+==========+ +| Synchronize | ``.tgz`` | ++---------------+----------+ +| Download | ``.zip`` | ++---------------+----------+ + +The use of ``.tgz`` for synchronization ensures efficient transfer between +management servers, while ``.zip`` is used for administrator downloads for +broad compatibility. + +Typical Workflows +^^^^^^^^^^^^^^^^^ + +Synchronize Extension Across Cluster +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Install or update the extension on one management server. +2. Call the ``syncExtension`` API or use **Sync Extension** in the UI. +3. Verify synchronization across all nodes. + +Download Extension for Backup or Migration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Call the ``downloadExtension`` API or use **Download Extension** in the UI. +2. Use the generated signed URL. +3. Download the ``.zip`` archive before the URL expires. + +Best Practices +^^^^^^^^^^^^^^ + +* Always synchronize extensions after modifying files on a single management + server in a clustered setup. +* Use Secondary Storage download mode when management servers are behind + private networks. +* Monitor management server logs for sync failures due to permission or + filesystem issues. +* Adjust ``extension.share.link.validity.interval`` according to your + operational security requirements. From bcc6d966969aaef1660647492a682b53046b3f57 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 12 Feb 2026 10:18:23 +0530 Subject: [PATCH 2/2] fix tables Signed-off-by: Abhishek Kumar --- .../adminguide/extensions/sync_download.rst | 39 +++++++++++-------- 1 file changed, 23 insertions(+), 16 deletions(-) diff --git a/source/adminguide/extensions/sync_download.rst b/source/adminguide/extensions/sync_download.rst index 20a8254bf9..4dcf9269dd 100644 --- a/source/adminguide/extensions/sync_download.rst +++ b/source/adminguide/extensions/sync_download.rst @@ -41,6 +41,8 @@ configure cache behavior, and secure download links. Share Endpoint Properties ~~~~~~~~~~~~~~~~~~~~~~~~~ +.. cssclass:: table-striped table-bordered table-hover + +----------------------+----------------------------------------------+---------------------------------------------+ | Property | Default Value | Description | +======================+==============================================+=============================================+ @@ -48,19 +50,19 @@ Share Endpoint Properties | | | feature. Must be ``true`` for extension | | | | downloads to function. | +----------------------+----------------------------------------------+---------------------------------------------+ -| share.base.dir | /share | Base directory from which files can be | +| share.base.dir | /share | Base directory from which files can be | | | | shared. If not explicitly set, the default | -| | | directory under the CloudStack user home | +| | | directory under the CloudStack user home | | | | is used. | +----------------------+----------------------------------------------+---------------------------------------------+ -| share.cache.control | public,max-age=86400,immutable | Cache-Control header value applied to | -| | | shared files. Controls browser/client | +| share.cache.control | public,max-age=86400,immutable | Cache-Control header value applied to | +| | | shared files. Controls browser/client | | | | caching behavior. | +----------------------+----------------------------------------------+---------------------------------------------+ -| share.secret | change-me | Secret key used to generate HMAC-signed | -| | | download links. It is strongly recommended | -| | | to change this value in production. If not | -| | | set, links will not be signed. | +| share.secret | change-me | Secret key used to generate HMAC-signed | +| | | download links. It is strongly recommended | +| | | to change this value in production. If not | +| | | set, links will not be signed. | +----------------------+----------------------------------------------+---------------------------------------------+ Notes: @@ -192,14 +194,17 @@ selected management server. Global Configuration Parameters -------------------------------- -+-----------------------------------------------+--------------------------------------------------------------+ -| Name | Description | -+===============================================+==============================================================+ -| extension.share.download.use.secondary.storage| If ``true``, forces extension downloads via Secondary | -| | Storage (SSVM). | -+-----------------------------------------------+--------------------------------------------------------------+ -| extension.share.link.validity.interval | Validity duration (in seconds) of the signed download URL. | -+-----------------------------------------------+--------------------------------------------------------------+ +.. cssclass:: table-striped table-bordered table-hover + ++------------------------------------------------+---------------+-----------------------------------------+ +| Name | Default Value | Description | ++================================================+===============+=========================================+ +| extension.share.download.use.secondary.storage | false | If ``true``, forces extension downloads | +| | | via Secondary Storage (SSVM). | ++------------------------------------------------+---------------+-----------------------------------------+ +| extension.share.link.validity.interval | 3600 | Validity duration (in seconds) of the | +| | | signed download URL. | ++------------------------------------------------+---------------+-----------------------------------------+ Events and Logging ------------------ @@ -215,6 +220,8 @@ These changes improve traceability and operational visibility. Archive Formats --------------- +.. cssclass:: table-striped table-bordered table-hover + +---------------+----------+ | Operation | Format | +===============+==========+