diff --git a/source/adminguide/extensions.rst b/source/adminguide/extensions.rst index 2c6b7308e0..e62ecca2c8 100644 --- a/source/adminguide/extensions.rst +++ b/source/adminguide/extensions.rst @@ -94,6 +94,8 @@ CloudStack provides built-in Orchestrator Extensions for Proxmox, Hyper-V, and M .. include:: extensions/inbuilt_extensions.rst +.. include:: extensions/sync_download.rst + .. include:: extensions/limitations.rst .. include:: extensions/troubleshooting.rst diff --git a/source/adminguide/extensions/sync_download.rst b/source/adminguide/extensions/sync_download.rst new file mode 100644 index 0000000000..4dcf9269dd --- /dev/null +++ b/source/adminguide/extensions/sync_download.rst @@ -0,0 +1,264 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information# + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + + +Synchronization and Download +============================ + +In clustered Apache CloudStack management server deployments, extension files +must remain consistent across all management nodes. This release introduces +built-in support for: + +* Synchronizing extension files across management servers +* Downloading extension files as an archive +* Secure file sharing via Secondary Storage (SSVM) when direct access to a + management server is not possible + +These capabilities are available via new APIs and UI actions. + +Share Endpoint Configuration +---------------------------- + +Extension download and synchronization workflows rely on the management +server's **share endpoint**, which enables controlled file sharing. + +The share endpoint is configured in ``server.properties`` and allows +administrators to enable or disable file sharing, define the base directory, +configure cache behavior, and secure download links. + +Share Endpoint Properties +~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. cssclass:: table-striped table-bordered table-hover + ++----------------------+----------------------------------------------+---------------------------------------------+ +| Property | Default Value | Description | ++======================+==============================================+=============================================+ +| share.enabled | true | Enables or disables the file sharing | +| | | feature. Must be ``true`` for extension | +| | | downloads to function. | ++----------------------+----------------------------------------------+---------------------------------------------+ +| share.base.dir | /share | Base directory from which files can be | +| | | shared. If not explicitly set, the default | +| | | directory under the CloudStack user home | +| | | is used. | ++----------------------+----------------------------------------------+---------------------------------------------+ +| share.cache.control | public,max-age=86400,immutable | Cache-Control header value applied to | +| | | shared files. Controls browser/client | +| | | caching behavior. | ++----------------------+----------------------------------------------+---------------------------------------------+ +| share.secret | change-me | Secret key used to generate HMAC-signed | +| | | download links. It is strongly recommended | +| | | to change this value in production. If not | +| | | set, links will not be signed. | ++----------------------+----------------------------------------------+---------------------------------------------+ + +Notes: + +* ``share.enabled`` must be ``true`` for extension downloads to function. +* If ``share.secret`` is configured, generated links are signed using HMAC. +* It is strongly recommended to replace the default ``share.secret`` value + in production environments. +* If ``share.secret`` is not configured, download links will not be signed. + +Extension Synchronization +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In a multi-node management server cluster, extension files are stored locally +on each management server. If an extension is installed or modified on one +node, it must be synchronized to peers. + +syncExtension API +~~~~~~~~~~~~~~~~~ + +The ``syncExtension`` API triggers synchronization of a selected extension +across all management servers in the cluster. + +How It Works +~~~~~~~~~~~~ + +1. The source management server: + + * Calculates checksums for all files in the extension directory. + * Packages the extension directory (or selected files) into a ``.tgz`` archive. + +2. The server sends a ``DownloadAndSyncExtensionFilesCommand`` to peer + management servers. + +3. Each peer management server: + + * Downloads the archive. + * Stages it temporarily. + * Extracts the files into the extension directory. + * Validates checksums. + * Cleans up temporary files. + +Checksum Validation +~~~~~~~~~~~~~~~~~~~ + +Checksums are calculated for all files in the extension directory and compared +during synchronization. This ensures: + +* File integrity +* Detection of missing or modified files +* Cluster-wide consistency + +UI Support +~~~~~~~~~~ + +In the CloudStack UI: + +1. Navigate to **Extensions** +2. Select an extension +3. Click **Sync Extension** + +This triggers synchronization across all management servers. + +Downloading Extension Files +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You can download an extension’s files as an archive for backup, inspection, +or migration. + +downloadExtension API +~~~~~~~~~~~~~~~~~~~~~ + +The ``downloadExtension`` API allows administrators to download the complete +extension directory as a ``.zip`` archive. + +How It Works +~~~~~~~~~~~~ + +1. The management server creates a ``.zip`` archive of the extension directory. +2. A secure share URL is generated. +3. The administrator downloads the archive using the signed URL. + +Secure Share URL +~~~~~~~~~~~~~~~~ + +Download URLs are: + +* Signed using HMAC +* Time-bound (expire automatically) +* Cleaned up after expiry + +The validity interval is controlled via the global configuration: + +.. code-block:: text + + extension.share.link.validity.interval + +This defines how long (in seconds) the signed URL remains valid before +automatic cleanup. + +Download via Secondary Storage (SSVM) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In some environments, a management server may not be directly accessible from +the administrator’s network. + +To support this scenario: + +* The archive can be staged on Secondary Storage. +* The signed URL points to the SSVM endpoint. +* The file is served via Secondary Storage instead of directly from the + management server. + +Global Configuration +~~~~~~~~~~~~~~~~~~~~ + +To force downloads via Secondary Storage: + +.. code-block:: text + + extension.share.download.use.secondary.storage = true + +When set to ``true``, extension downloads are always served via Secondary +Storage (SSVM). + +When set to ``false`` (default), downloads are served directly from the +selected management server. + +Global Configuration Parameters +-------------------------------- + +.. cssclass:: table-striped table-bordered table-hover + ++------------------------------------------------+---------------+-----------------------------------------+ +| Name | Default Value | Description | ++================================================+===============+=========================================+ +| extension.share.download.use.secondary.storage | false | If ``true``, forces extension downloads | +| | | via Secondary Storage (SSVM). | ++------------------------------------------------+---------------+-----------------------------------------+ +| extension.share.link.validity.interval | 3600 | Validity duration (in seconds) of the | +| | | signed download URL. | ++------------------------------------------------+---------------+-----------------------------------------+ + +Events and Logging +------------------ + +The following improvements are included: + +* Dedicated events for sync and download operations +* Improved logging for archive creation, transfer, extraction, and cleanup +* Automatic cleanup of temporary share files after expiry + +These changes improve traceability and operational visibility. + +Archive Formats +--------------- + +.. cssclass:: table-striped table-bordered table-hover + ++---------------+----------+ +| Operation | Format | ++===============+==========+ +| Synchronize | ``.tgz`` | ++---------------+----------+ +| Download | ``.zip`` | ++---------------+----------+ + +The use of ``.tgz`` for synchronization ensures efficient transfer between +management servers, while ``.zip`` is used for administrator downloads for +broad compatibility. + +Typical Workflows +^^^^^^^^^^^^^^^^^ + +Synchronize Extension Across Cluster +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Install or update the extension on one management server. +2. Call the ``syncExtension`` API or use **Sync Extension** in the UI. +3. Verify synchronization across all nodes. + +Download Extension for Backup or Migration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Call the ``downloadExtension`` API or use **Download Extension** in the UI. +2. Use the generated signed URL. +3. Download the ``.zip`` archive before the URL expires. + +Best Practices +^^^^^^^^^^^^^^ + +* Always synchronize extensions after modifying files on a single management + server in a clustered setup. +* Use Secondary Storage download mode when management servers are behind + private networks. +* Monitor management server logs for sync failures due to permission or + filesystem issues. +* Adjust ``extension.share.link.validity.interval`` according to your + operational security requirements.