Add logs for storage pools reordering by julien-vaz · Pull Request #10419 · apache/cloudstack

julien-vaz · 2025-02-18T14:12:26Z

Description

Since the storage pool reordering process was lacking logs for troubleshooting, some debug level log messages were added and trace level log messages were changed to the debug level. Also, the log messages were rewritten according to the new Log4j2 syntax.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

A volume was created;
Its disk offering was changed;
It was verified that the log messages were properly being displayed.

boring-cyborg · 2025-02-18T14:12:31Z

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

In case of a new feature add useful documentation (raise doc PR at https://github.com/apache/cloudstack-documentation)
Be patient and persistent. It might take some time to get a review or get the final approval from the committers.
Pay attention to the quality of your code, ensure tests are passing and your PR doesn't have conflicts.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Issues, Mailing list and Slack.
Be sure to read the CloudStack Coding Conventions.
Apache CloudStack is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@cloudstack.apache.org (https://cloudstack.apache.org/mailing-lists.html)
Slack: https://apachecloudstack.slack.com/

codecov · 2025-02-18T15:14:51Z

Codecov Report

❌ Patch coverage is 20.00000% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.62%. Comparing base (7324ef4) to head (7be068d).

Files with missing lines	Patch %	Lines
...torage/allocator/AbstractStoragePoolAllocator.java	20.00%	24 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               4.22   #10419      +/-   ##
============================================
- Coverage     17.62%   17.62%   -0.01%     
+ Complexity    15668    15666       -2     
============================================
  Files          5917     5917              
  Lines        531255   531253       -2     
  Branches      64951    64947       -4     
============================================
- Hits          93639    93638       -1     
  Misses       427077   427077              
+ Partials      10539    10538       -1

Flag	Coverage Δ
uitests	`3.70% <ø> (ø)`
unittests	`18.69% <20.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java

DaanHoogland

one general remark on these changes; I see only trace and debug logs, but if axtra logs are really needed I would expect at least hight level logging to be added (info and higher) were considerations made to do this? (DEBUG should not be on in production environments in principle, unless trouble shooting is going on)

julien-vaz · 2025-03-05T16:41:13Z

Alright, I've just changed the level of logs on reordering methods, @DaanHoogland

DaanHoogland

makes mostly sense, just one info i'd like to question.

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java

julien-vaz · 2025-03-06T17:32:14Z

@blueorangutan package

blueorangutan · 2025-03-06T17:34:04Z

@julien-vaz a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-03-06T18:46:43Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12691

DaanHoogland

clgtm

bernardodemarco

clgtm

github-actions · 2025-05-16T16:35:37Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

JoaoJandre · 2025-06-02T20:03:39Z

@julien-vaz could you fix the conflicts?

julien-vaz · 2025-06-04T17:21:28Z

@julien-vaz could you fix the conflicts?

Sure!

sureshanaparti · 2025-06-05T09:31:34Z

@blueorangutan package

blueorangutan · 2025-06-05T09:32:04Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-05T11:54:33Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 13617

DaanHoogland · 2025-06-09T12:59:46Z

@blueorangutan test

blueorangutan · 2025-06-09T13:02:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-06-10T05:55:21Z

[SF] Trillian test result (tid-13481)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57861 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10419-t13481-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland · 2025-06-11T17:43:17Z

@sureshanaparti , do you think we need anymore testing on this? (/me not convinced)

julien-vaz · 2025-06-23T14:24:55Z

I've just built the packages locally, so I'm triggering the Blue Orangutan again.

julien-vaz · 2025-06-23T14:25:09Z

@blueorangutan package

blueorangutan · 2025-06-23T14:26:04Z

@julien-vaz a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

rajujith · 2026-01-12T08:42:47Z

@julien-vaz Since this is for the 4.22.1 release, could you retarget the PR to the 4.22 branch?

borisstoyanov

LGTM, based on code review and BO tests. @sureshanaparti could you check the branch and merge please

Copilot

Pull request overview

This PR enhances observability of storage pool reordering by adding and updating log messages and migrating them to Log4j2-style parameterized logging.

Changes:

Replaced String.format-based logging with Log4j2-style {} placeholders throughout AbstractStoragePoolAllocator.
Added and elevated several log statements (including to INFO level) around pool reordering by capacity, number of volumes, and disk provisioning type.
Adjusted logging for filtering and compatibility checks on storage pools to provide more structured and detailed context for troubleshooting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-26T09:19:05Z

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java

                    reorderedPools.add(preferredIndex++, pool);
                }
            }
+            logger.debug("Reordered list of pools by disk provisioning type [{}]: [{}]", diskProfile.getProvisioningType(), pools);


This debug statement logs the original pools list even though the method has just built a reorderedPools list and is about to return it, which makes the message misleading when troubleshooting reordering by disk provisioning type. Consider logging the reordered list instead so the log output reflects the actual ordering that will be used.

Suggested change

logger.debug("Reordered list of pools by disk provisioning type [{}]: [{}]", diskProfile.getProvisioningType(), pools);

logger.debug("Reordered list of pools by disk provisioning type [{}]: [{}]", diskProfile.getProvisioningType(), reorderedPools);

Copilot · 2026-01-26T09:19:05Z

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java

        if (pools == null) {
-            logger.trace("There are no pools to reorder; returning null.");
+            logger.info("There are no pools to reorder.");
            return null;
        }
-        if (logger.isTraceEnabled()) {
-            logger.trace(String.format("reordering %d pools", pools.size()));
-        }
+        logger.info("Reordering [{}] pools", pools.size());


These log statements (and similar ones added below) have been raised to INFO level, which means every pool reordering attempt will now be emitted at INFO instead of DEBUG/TRACE and can significantly increase log volume in busy environments. This also goes beyond the PR description (which mentioned adding DEBUG logs and promoting TRACE to DEBUG), so please confirm that INFO is the intended level for these high-frequency paths or consider keeping them at DEBUG while still adopting the new Log4j2 placeholder syntax.

borisstoyanov · 2026-01-27T08:56:57Z

@julien-vaz can you please resolve the conflicts

Substitui por nova sintaxe do Log4j

…ocator/AbstractStoragePoolAllocator.java Co-authored-by: dahn <daan.hoogland@gmail.com>

erikbocks

clgtm

erikbocks · 2026-02-12T11:53:15Z

Hello, @borisstoyanov and @rajujith

The conflicts were solved, can we move forward with this PR? I think it is ready to be merged.

DaanHoogland · 2026-02-12T13:02:40Z

@blueorangutan package

blueorangutan · 2026-02-12T13:04:04Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

sureshanaparti · 2026-02-12T13:40:32Z

@julien-vaz @erikbocks can you check the build errors.

Error:  testregisterBackupForVMwithOutSsid(org.apache.backup.networker.NetworkerClientTest)  Time elapsed: 0.387 s  <<< FAILURE!
...
Error:    NetworkerClientTest.testregisterBackupForVMwithOutSsid:637 No requests exactly matched. Most similar request was:  expected:<

blueorangutan · 2026-02-12T14:13:58Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16813

erikbocks · 2026-02-12T15:38:40Z

@sureshanaparti I runned the tests locally and no problem was found. I will trigger the build again.

boring-cyborg bot added the component:storage label Feb 18, 2025

bernardodemarco reviewed Mar 1, 2025

View reviewed changes

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java Outdated Show resolved Hide resolved

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java Outdated Show resolved Hide resolved

DaanHoogland reviewed Mar 3, 2025

View reviewed changes

DaanHoogland reviewed Mar 6, 2025

View reviewed changes

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java Outdated Show resolved Hide resolved

DaanHoogland approved these changes Mar 7, 2025

View reviewed changes

bernardodemarco approved these changes Apr 7, 2025

View reviewed changes

github-actions bot added the status:has-conflicts label May 16, 2025

julien-vaz force-pushed the add-logs-storage-pool-reordering branch from 53c59bf to 04e69aa Compare June 4, 2025 21:26

github-actions bot removed the status:has-conflicts label Jun 4, 2025

sureshanaparti added this to the 4.21.0 milestone Jun 5, 2025

sureshanaparti added this to Apache CloudStack 4.21.0 Jun 5, 2025

DaanHoogland marked this pull request as draft October 10, 2025 10:12

harikrishna-patnala modified the milestones: 4.22.0, 4.22.1 Nov 12, 2025

harikrishna-patnala removed this from Apache CloudStack 4.22.0 Nov 12, 2025

rajujith added this to Apache CloudStack 4.22.1 Jan 5, 2026

sureshanaparti moved this to In Progress in Apache CloudStack 4.22.1 Jan 7, 2026

DaanHoogland changed the base branch from main to 4.22 January 12, 2026 09:58

borisstoyanov marked this pull request as ready for review January 26, 2026 09:09

Copilot AI review requested due to automatic review settings January 26, 2026 09:09

Copilot started reviewing on behalf of borisstoyanov January 26, 2026 09:09 View session

borisstoyanov approved these changes Jan 26, 2026

View reviewed changes

borisstoyanov self-assigned this Jan 26, 2026

Copilot AI reviewed Jan 26, 2026

View reviewed changes

Julien Hervot de Mattos Vaz and others added 3 commits February 12, 2026 08:38

Adição de logs na reordenação de storage pools

d0f35d9

Substitui por nova sintaxe do Log4j

Apply suggestions

ee62f3c

Update engine/storage/src/main/java/org/apache/cloudstack/storage/all…

de58344

…ocator/AbstractStoragePoolAllocator.java Co-authored-by: dahn <daan.hoogland@gmail.com>

erik-bock-silva force-pushed the add-logs-storage-pool-reordering branch from b69581a to de58344 Compare February 12, 2026 11:39

github-actions bot removed the status:has-conflicts label Feb 12, 2026

log reordered pools list instead of pools list

7be068d

erikbocks approved these changes Feb 12, 2026

View reviewed changes

erikbocks closed this Feb 12, 2026

erikbocks reopened this Feb 12, 2026

	logger.debug("Reordered list of pools by disk provisioning type [{}]: [{}]", diskProfile.getProvisioningType(), pools);
	logger.debug("Reordered list of pools by disk provisioning type [{}]: [{}]", diskProfile.getProvisioningType(), reorderedPools);

Conversation

julien-vaz commented Feb 18, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

How Has This Been Tested?

Uh oh!

boring-cyborg bot commented Feb 18, 2025

Uh oh!

codecov bot commented Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

julien-vaz commented Mar 5, 2025

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

julien-vaz commented Mar 6, 2025

Uh oh!

blueorangutan commented Mar 6, 2025

Uh oh!

blueorangutan commented Mar 6, 2025

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

bernardodemarco left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 16, 2025

Uh oh!

JoaoJandre commented Jun 2, 2025

Uh oh!

julien-vaz commented Jun 4, 2025

Uh oh!

sureshanaparti commented Jun 5, 2025

Uh oh!

blueorangutan commented Jun 5, 2025

Uh oh!

blueorangutan commented Jun 5, 2025

Uh oh!

DaanHoogland commented Jun 9, 2025

Uh oh!

blueorangutan commented Jun 9, 2025

Uh oh!

blueorangutan commented Jun 10, 2025

Uh oh!

DaanHoogland commented Jun 11, 2025

Uh oh!

julien-vaz commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

julien-vaz commented Jun 23, 2025

Uh oh!

blueorangutan commented Jun 23, 2025

Uh oh!

rajujith commented Jan 12, 2026

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

borisstoyanov commented Jan 27, 2026

codecov bot commented Feb 18, 2025 •

edited

Loading

julien-vaz commented Jun 23, 2025 •

edited

Loading