Delegate release notes
These release notes describe recent changes to Harness Delegate.
- Progressive deployment: Harness deploys changes to Harness SaaS clusters on a progressive basis. This means that the features described in these release notes may not be immediately available in your cluster. To identify the cluster that hosts your account, go to your Account Overview page in Harness. In the new UI, go to Account Settings, Account Details, General, Account Details, and then Platform Service Versions.
- Security advisories: Harness publishes security advisories for every release. Go to the Harness Trust Center to request access to the security advisories.
- More release notes: Go to Harness Release Notes to explore all Harness release notes, including module, delegate, Self-Managed Enterprise Edition, and FirstGen release notes.
Delegate Base Image Migration
Harness is planning to update the base image for its Delegate from redhat/ubi8-minimal:8.10
to redhat/ubi9-minimal:9.4
, as UBI-8 reached end-of-life on May 31st, 2024. No further updates, patches, or fixes will be provided for UBI-8, so this migration ensures continued security and compatibility. This change will take effect starting January 6, 2025.
Key Updates with UBI9 Migration:
-
Microdnf Command Update: When installing or removing any tool via the
microdnf
command, the confirmation option-y
is now required.- Example:
microdnf install wget -y
- Example:
-
Tool Availability:
curl
is already included inubi9-minimal
, so manual installation is no longer necessary.
Action Required: If you use an init_script
or a custom Dockerfile for your Delegate image, please incorporate these updates to avoid compatibility issues.
For more details on UBI9, please refer to the UBI9 Release Notes.
Added a critical security fix in harness secret manager for handling identities with CD workflows. If you are running delegates version below 799xx and using Terraform/Terragrunt features, upgrade to delegate version 799x or above immediately. Go to the Delegate automatic upgrades and expiration policy to update the delegates.
Delegate version 24.08.83702 is affected due to rendering logic of Kubernetes Manifest in certain cases only. If you are using this version, please upgrade to version 24.08.83704 to resolve the issue
Certain delegate versions (24.07.83608
, 24.07.83607
, 24.07.83606
, 24.07.83605
) are affected due to baked-in AMD64 client binaries on ARM64 architecture, despite building a multiarch image. If you are using any of these versions on ARM64 architecture, please upgrade to version 24.07.83609
or 24.07.83609.minimal
to resolve the issue.
If you have blocked Stackdriver logs using firewall rules, upgrade your delegates to version 24.06.83304 or later.
Deprecation notice
This is an End of Support (EOS) notice for the Delegate-Legacy image type. This image type reached End of Support (EOS) as of January 31, 2024.
End of Support means the following:
- Harness Support will no longer accept support requests for the Delegate-Legacy image type in both Harness FirstGen and Harness NextGen (including Harness Self-Managed Enterprise Edition (SMP)).
- Security fixes will still be addressed.
- Product defects will not be addressed.
Follow the below steps to upgrade Delegate-Legacy to Delegate image
- Download new yaml from Harness by keeping the same name as the previous delegate
- Check if the existing delegate has any tags/selector, if yes then add them in DELEGATE_TAGS
- Compare the permissions given to the legacy delegate in their yaml and give the same permissions to new delegates
- Check if custom image is used, if yes then build a new image with immutable delegate as base image and override the account setting to point to that image
- Ensure that auto upgrade is enabled for Kubernetes delegates
- Our delegate yaml ships with default HPA of min and max replicas to be 1, adjust the desired number of replicas in HPA
- Deploy the new yaml and see new replicas coming under the same delegate
- Scale down the old stateful set and verify that everything is correct
Harness has updated the delegate expiration policy to 6 months with a 2-month EOL upgrade period.
Six months after a delegate image is released, the delegate reaches End of Support (EOS). Eight months after a delegate image is released, the delegate is End of Life (EOL). Delegates expire if not upgraded 6 months after the image is released. If delegates are past their EOS date, Harness does not support them. Expired delegates might not work as intended. For issues with expired delegates, Harness Support will request that you upgrade your delegate(s).
For more information, go to Delegate expiration support policy.
January 2025
Version 25.01.84800
New features and improvements
- Added a new metric on the delegate side to track the number of times the delegate WebSocket reconnects. This metric,
io_harness_custom_metric_delegate_reconnected_total
, can be used to set alerts for frequent reconnections, helping identify potential issues with the delegate and enabling you to seek further assistance from Harness if needed. (PL-48535)
December 2024
Version 24.11.84503
Fixed issues
- Fixed an issue that prevented users from retrieving secrets from the HashiCorp Vault when the path contained special characters. The solution involved enhancing support for "dots" in dynamic secret reference expressions, ensuring seamless retrieval of such secrets. (PL-58771, ZD-73710, ZD-73724)
New features and improvements
- Updated
“org.bouncycastle:bcpkix-jdk18on”
to version 1.78 and removed“org.bouncycastle:bcprov-jdk15on”
(version 1.70) from the Delegate. (PL-58474)
Version 24.11.84311
Hotfix
- Previously, when there was deployment failure in TAS Rolling deployment, secrets were printed in the delgate logs. This issue has now been fixed. (CDS-105208)
Version 24.11.84310
Hotfix
- Increase Azure Web App Http client ReadTimeout duration to 230 Seconds. (CDS-104813)
Version 24.10.84106
Hotfix
- Previously, when users encountered the Too many files open error while running pipelines, insufficient logs made it challenging to debug the issue. Additional logs have been added to help triage and identify the root cause of this issue. (PIPE-23686, ZD-72845,73732)
Version 24.11.84502
Hotfix
- Removed CVE-2024-29857 & CVE-2024-30172 from delegate image. (PL-58901)
Version 24.11.84501
- Implemented functionality to expose Custom CF CLI variables while executing any CF CLI commands
Version 24.07.83407
Hotfix
- Encoding Nexus 2 URI to support downloading artifacts which contains special characters in artifact version. (CDS-102807)
Version 24.11.84500
New features and enhancements
-
Enhanced AWS Secrets Manager integration to support secret updates using the
secretsmanager:PutResourcePolicy
permission. The appropriate request type (UpdateSecret
orPutSecretValue
) is now determined based on theusePutSecret
flag in the connector configuration. (PL-58652) -
Upgraded Spring Framework to version
6.1.x
, along with updates to dependencies for improved compatibility and security. (PL-58254) -
Resolved a high-severity vulnerability (CVE-2024-7254) in the Delegate by upgrading
protobuf-java
to version3.25.5
. (PL-57351, ZD-70765) -
Upgraded the Java version to
17.0.11_9-jre-ubi9-minimal
in the Delegate base image to address security vulnerabilities, includingCVE-2023-22041
. (PL-55499) -
Upgraded
com.nimbusds_nimbus-jose-jwt
to version9.37.2
to addressCVE-2023-52428
. (PL-51347)
November 2024
Version 24.11.84309
New features and enhancements
- Implemented functionality to support groupByResource for the Datadog Health Source in Continuous Verification. (CDS-100367)
Version 24.11.84308
- The existing behaviour does not support returning the full, unredacted manifest in an encrypted format as the dry run output. With this fix, the full manifest is encrypted and returned as output, with no redactions. (CDS-103383)
Version 24.11.84307
- The customer encountered a pipeline failure when they enabled the CDS_K8S_CUSTOM_YAML_PARSER feature and used a YAML manifest with parameters supported by the 21.x.x version of the Kubernetes Java SDK. The issue arose due to a YAML parsing error. (CDS-104066)
Version 24.10.84205-ubi9-beta
Early release (Beta release).
- Upgrading redhat/ubi8-minimal to redhat/ubi9-minimal for testing purpose. This image can have issues as this is a Beta image and not a GA image.
Version 24.11.84306
Fixed issues
-
The delegate name is now displayed in the UI whenever a connector test fails, provided the validation task was acquired by a delegate. This enhancement offers better visibility into which delegate handled the task during troubleshooting. (PL-56483, ZD-64425)
-
Fixed Azure WebApp deployment pipeline failures for specific connectors configured with ignoreTestConnection. (CDS-103533)
-
Fixed the instance synchronization issue for Azure web applications. (CDS-103224)
New features and enhancements
-
Added a new scope query parameter to the
listDelegates
endpoint. When set to true, this parameter enables listing delegates across hierarchical scopes (Account, Org, Project). By default, scope is set to false. (PL-57724) -
Upgraded the base image for
delegate
,delegate-minimal
,ci-addon
, andlite-engine
fromredhat/ubi8-minimal:8.8
toredhat/ubi8-minimal:8.10
. This update enhances security and compatibility with the latest UBI version. (PL-58062) -
Updated the
delegate/rings
API to return the immutable delegate version instead of the legacy delegate version. Additionally, theconnected-ratio-with-primary
andconnected-delegate-ratio
APIs have been removed. (PL-57518)
Version 24.10.84200
Fixed issues
- Removed restrictions on the Delegate metrics API endpoint, allowing requests with any Content-Type header. This update supports improved compatibility with monitoring tools like Dynatrace. (PL-57704, ZD-71319)
New features and enhancements
- Set limits on the number of delegates and delegate tokens allowed per account and per scope. The current limit is set to 10,000. (PL-56296)
October 2024
Version 24.10.84105
New features and enhancements
- Add support for k8s sidecar containers
Version 24.10.84104
New features and enhancements
- Implemented a limit on the number of delegates and delegate tokens per account and per scope. The maximum number of delegate tokens is now set to 10,000 to ensure better management and scalability. (PL-56296)
Fixed issues
- Improved error messaging for the
<+secrets.getValue(secretlocation)>
expression to provide clearer feedback when a secret is not found. The updated message now states, "The secret has not been found," and includes the full computed path for better troubleshooting. (PL-51900, ZD-65130, ZD-69181)
Version 24.09.83909
Hotfixes
-
Improved logging, error handling and force shutdown for stuck cases in winrm script for collecting output variables. These changes are behind a delegate environment variable
ENV_VARS_COLLECTOR_EXPLICIT_EXIT
. Also delegate environment variableWINRM4J_LOG_LEVEL
forio.cloudsoft.winrm4j
logging level has been added. (CDS-101843) -
With this change entire k8s dry manifest output yaml won't be sanitized. Only config map and secrets kind blocks would be sanitised unless `CDS_K8S_SANITIZE_COMPLETE_DRY_RUN_STEP_OUTPUT`` feature flag is switched on. (CDS-101686)
-
Jira steps will now ignore unsupported fields which reading a jira ticket. (CDS-101162)
Version 24.09.83906
Hotfixes
- ASG step will not delete all tags and create tags but instead only remove those tags which are not present while running pipleine. (CDS-101285)
September 2024
Version 24.08.83805
Hotfixes
- WinRM shell script steps now support logs more than 5 hours till maximum of step timeout or 1 day (CDS-101408).
Version 24.09.83905
Hotfixes
- Updated the identifier so that the output obtained from the PowerShell command is parsed correctly (CDS-100036).
Version 24.08.83803
Hotfixes
- Fixed an issue where the secrets will no longer get exposed in Kubernetes Dry Run Step even if they are placed in ConfigMap.
Version 24.09.83900
Fixed issues
-
Enhanced webhook notification handling to support secrets in headers, enabling proper decryption of Authorization and other header values stored in the Harness Secret Manager. This ensures seamless webhook triggering without requiring hardcoded values. (PL-55319, ZD-65913)
-
Fixed an issue where the AWS Secret Manager validation was failing due to regions being passed instead of full URLs, causing connectivity errors in delegate logs. The region is now correctly converted to a URL, preventing perpetual task failures. (PL-55740, ZD-67142, ZD-67150)
August 2024
Version 24.08.83802
New features and enhancements
- Upgraded the
dnsjava
library to version3.6.0
to address CVE-2024-25638, which involved potential security vulnerabilities in DNS query responses. (PL-55721, ZD-63383, ZD-68810)
Version 24.07.83611, 24.08.83705
Hotfix
- Removed unnecessary env expansion and added url_encoding to encode special characters from proxy when curl connectivity pre-check is enabled (PL-56623).
Version 24.08.83704
Hotfixes
- Ensure kubernetes secrets are typecasted to Java strings internally before log sanitization. Earlier this was causing ClassCastException for some kubernetes manifests (CDS-100389).
- Updated sensitive log in WinRM deployment to DEBUG level to ensure sensitive data is not leaked (CDS-100046).
Version 24.07.83609
Hotfix
- Modified the default value handling for built-in Docker environment variables for
TARGETPLATFORM
Version 24.08.83701
New features and enhancements
-
Enhanced AppRole token cache for HashiCorp Vault: Updated the cache key calculation to include secretId and approleId. This change fixes a problem where tokens were not being refreshed correctly. Now, the cache accurately reflects the latest credentials, ensuring secure and reliable token management. (PL-55567, ZD-65493)
-
Added proxy configuration support for external notification channels in SMP. To address issues faced by customers who operate in air-gapped environments, we've introduced proxy settings for the platform service. By updating the override file with proxy details, notifications via MS Teams and Slack will now function correctly even when behind a proxy. This feature is available in SMP version 0.19.0. (PL-48415, ZD-59707, ZD-62139)
Fixed issues
-
The delegate initialization process has been moved from a background thread to the start of application. This change addresses issues with health check failures during startup by ensuring that delegate registration, websocket establishment, and heartbeat scheduling are completed before health checks are performed. (PL-55905, ZD-67667)
-
Resolved issue with Rollout deployment logs where logs were not available or expandable. This problem, caused by a race condition between stream closure and log dispatching, has been fixed. Logs will now display correctly even under heavy load. (PL-55512, ZD-66330)
Version 24.07.83608
- Separated the LDAP settings between CG and NG. With this feature, the CG LDAP upgrade to NG LDAP, and CG and NG LDAP settings now operate independently. This feature is behind the feature flag
PL_ENABLE_NG_LDAP_SETTINGS
. To enable this feature, please contact Harness Support. (PL-56167)
Version 24.08.83306
Hotfix
- Sensitive secrets were logged in plain text in
delegate.log
due to the use ofsecrets.getValue
in environment variables. The logging level for these events has been changed fromerror
todebug
to prevent exposure of secrets. (CI-13785, ZD-68120)
Version 24.07.83607
Hotfix
- Sensitive secrets were logged in plain text in
delegate.log
due to the use ofsecrets.getValue
in environment variables. The logging level for these events has been changed fromerror
todebug
to prevent exposure of secrets. (CI-13785, ZD-68120)
Version 24.07.83406
Hotfix
- Sensitive secrets were logged in plain text in
delegate.log
due to the use ofsecrets.getValue
in environment variables. The logging level for these events has been changed fromerror
todebug
to prevent exposure of secrets. (CI-13785, ZD-68120)
July 2024
Version 24.07.82906
Hotfix
- Rollout deployment logs were not available and could not be expanded. Although the deployment was working, the logs were not displaying. The issue has been addressed by ensuring that logs will be shown even on a heavily loaded delegate. (PL-55512, ZD-66330)
Version 24.07.83404
New features and enhancements
- Modified the unique index for delegate token names. The default token name in each scope will now be
default_token
instead ofdefault_token_org/project
. This change applies only to new projects and organizations; existing projects and organizations will retain their current token names. (PL-51151)
Version 24.07.83205
Hotfix
-
When the feature flag
CDS_PERFORM_SHELL_SCRIPT_HOST_CAPABILITY
is enabled, Shell script steps will perform host capability checks. (CDS-97512, ZD-66326, ZD-66349) -
Script executions failed during the Command step for WinRM deployments with a Kerberos auth type when environment variables contained the characters
\v
,\b
, or\f
. Now, when the feature flagCDS_ESCAPE_ENV_VARS_FOR_WINRM_KERBEROS
is enabled, the environment variables will be escaped and script execution will pass. (CDS-97690)
June 2024
Version 24.07.82905
Hotfix
- Reduced the time for missing heartbeats for delegates before the liveness probe fails from 15 mins to 5 mins. (PL-52037)
Version 24.06.83304
Fixed issues
-
Kubernetes services were created during the startup of the delegate, causing the IP pool to be exhausted for NAB. The delegate has been updated to prevent the creation of Kubernetes services upon startup, resolving the issue with IP pool exhaustion. (PL-51550)
-
Delegates were running out of memory due to frequent connectivity checks. Optimized the connectivity check process to reduce memory usage, preventing the delegate from running out of memory. (PL-51418, ZD-63705)
-
When trying to resolve the expressions in the File Store scripts, Harness encountered a self referencing expression. Due to this condition, the resources associated with two Harness services were exhausted. A code change fixed this issue by preventing such pipeline executions. (PIPE-19585, ZD-64579, ZD-64580)
Version 24.06.83203
Fixed issues
- Delegate logs were displaying entire bearer tokens when using the IDP Kubernetes connector. Added log sanitization to delegate logs to mask commonly used secret patterns. These patterns can be extended per-use-case by adding them to the
/opt/harness-delegate/sanitize-patterns.txt
file inside the delegate. (PL-50889, ZD-64069)
Version 24.06.83004
Hotfix
- Secrets were being printed in plain text when using a custom secret manager, exposing sensitive information. Implemented masking of the
script
field in the custom secret manager to prevent logging of secrets used within the script. (PL-51535, ZD-64069)
Version 24.06.83003
Hotfix
- Resolved an issue with missing labels for Karpenter-managed nodes. (CCM-18139)
May 2024
Version 24.05.82711
Hotfix
- Resolved an issue where delegates created Kubernetes services when starting up. (PL-51548, PL-51550, ZD-64345)
Version 24.05.83001
New features and enhancements
- Added support for proxies via Secure Connect for GitHub App connectors. (CI-12130, ZD-61883)
Version 24.05.82904
Hotfix
- Tanzu steps will resolve the PCF CLI plugins path by checking the
HOME
environment variable. (CDS-95794, ZD-61882)
Version 24.05.82205
Hotfix
- Delegates will now include memory resource statistics in their logs, providing valuable additional insight for troubleshooting memory-related issues. (PL-51027)
Version 24.05.82903
Hotfix
- Resolved an issue with the Google artifact registry trigger of a pipeline when using a GCP connector with OIDC authentication. (CDS-96627, ZD-62986)
Version 24.05.82902
Hotfix
- Resolved an issue with the delegate health endpoint, enabling the delegate to perform several websocket reconnection attempts before Kubernetes evicts the pod. (PL-50540, ZD-59551, ZD-62207)
Version 24.04.82901
Fixed issues
-
Delegates with mTLS enabled were able to send a heartbeat to Harness Manager despite being configured with a non-agent endpoint. Resolved this by ensuring the
isNg
flag is correctly propagated when delegates send heartbeats to Harness Manager. (PL-48891, ZD-60974) -
Intermittent socket timeout exceptions occurred in running pipelines due to secret decryption failures, triggering unnecessary re-broadcasts on the delegate side. Resolved the issue of intermittent secret decryption failures within pipelines, ensuring stable and uninterrupted pipeline execution. (PL-47940, ZD-58006)
-
Local login was failing for users assigned admin permissions via a user group. The method to verify if a user is an account admin only considered direct user assignments and did not account for user group roles. Revised the validation process to include both user and user group assignments when checking for admin status. Now, to be recognized as an admin, users must have the specific role assignments outlined below; assigning the
_account_admin
role alone is no longer sufficient for admin rights. (PL-47632)- Role:
_account_admin
. - Resource-group:
_all_resources_including_child_scopes
,_all_account_level_resources
.
- Role:
April 2024
Version 24.04.82804
Fixed issues
-
The delegate task rejection metric was designed to reflect tasks rejected by a delegate due to system-related reasons (such as lack of resources or exceeding the limit of parallel tasks) but did not include specific details like
taskType
ortask ID
. We have enhanced the task rejection metrics by addingtaskType
andtaskId
labels. (PL-48488) -
Users were being logged out when testing a Git connector with invalid credentials due to the Git client's 401 response being propagated to the UI. We have implemented error handling to convert a 401 response from the test connection step to a 400, while preserving the original error message, preventing unintended user logouts. (PL-47753, ZD-58629)
-
2FA reset emails failed to display the QR code properly due to the recent deprecation of Google APIs. The method for generating QR codes has been updated, resolving the issue and ensuring QR codes are now correctly included in 2FA reset emails. (PL-48980, ZD-61314, ZD-61420, ZD-61486)
Version 24.04.82707
New features and enhancements
-
Docker delegate images are no longer pushed to
app.harness.io/registry
. To pull images, usegcr.io/gcr-prod/harness/delegate:<IMAGE_TAG>
. (PL-46947) -
We've added an optional registry mirror configuration for delegate
upgrader
. If you use Docker pull through registry cache (https://docs.docker.com/docker-hub/mirror/
), you can configureupgrader
to use an optional registry mirror for your delegate images. For more information, go to Configure an optional registry mirror for delegate images. (PL-47920, ZD-59005)
Fixed issues
-
Slack channel notifications failed due to an error related to explicitly setting the Host header as
hooks.slack.com
. We have removed the explicit Host header setting to support both Slack-specific webhook URLs and regular URLs, resolving the issue. (PL-47914) -
In SCIM, creating a new user with special characters in their name failed, preventing the user from being added to Harness and resulting in discrepancies in user group membership between the Identity Provider and Harness. The name of a user will be sanitized if it does not follow Harness naming conventions during user addition flows. (PL-47614)
-
Builds triggered by Bitbucket Server push events had incorrect date information in the build history. This issue occurred due to missing date information in the
commits
object returned by the Bitbucket Server API. (CI-11556, ZD-58798) -
Delegate utilization metrics failed to decrease below a set threshold, even when rejecting all tasks. To solve this, memory-based threshold checks have been removed from the delegate due to functional discrepancies. (PL-48781, ZD-60713)
Version 24.04.82705
Hotfix
- Added support for network load balancers in ASG Blue Green deployments. (CDS-95510, ZD-60182)
Version 24.04.82603
Hotfix
- Added additional retries on failures when verifying Docker images during CD deployments. (CDS-93180, ZD-58933, ZD-59370, ZD-60138)
March 2024
Version 24.03.82601
Hotfix
- Added multiple log lines for debugging an issue. (CDS-93910)
Version 24.03.82600
New features and enhancements
- In the recent update to
ng-manager
version 1.28.0, we have implemented enhancements to the validation mechanism for secret identifiers. We now provide more flexibility and precision in validating secret identifiers, particularly regarding hyphen usage. While previously disallowed, secret identifiers can now contain hyphens. However, there are specific rules governing their usage. Hyphens are now permitted anywhere in the secret identifier, including at the end of the string. The updated validation allows for multiple occurrences of hyphens within the secret identifier. Secret identifiers cannot start with a hyphen, following best practices. (PL-46959)
Fixed issues
-
The delegate metrics endpoint
/api/metrics
had its content type set asapplication/json
, causing scraping issues with certain versions of Prometheus due to content type incompatibility. Attempts to switch to text/plain resulted in a 406 response code. We have revised the endpoint to deliver metrics inplainText
. You can now specify the desired content formatplainText
orJSON
by setting the "Accept" header in your request, ensuring broader compatibility with different Prometheus versions. (PL-46976, ZD-57489) -
Fixed an issue where Bitbucket connectors with API access enabled sometimes became unresponsive. (CDS-93298, ZD-56619, ZD-58844, ZD-59381)
-
Setting up a monitored service using cloud metrics from the Google Cloud Operations health source was unable to list dashboards to build query. (CDS-92355)
Version 24.03.82505
Hotfix
- The
ap-south-2
region is now supported for use with AWS Secrets Manager. (CDS-92541, ZD-58686)
Version 24.03.82502
New features and enhancements
-
Introduced separate environment variables to manage delegate resource thresholds for CPU and Memory when dynamic handling is enabled. Use
CPU_USAGE_THRESHOLD
for CPU control (default: no limit). UseMEMORY_USAGE_THRESHOLD
for memory control (default: 80%). If you are usingRESOURCE_USAGE_THRESHOLD
(deprecated), it exclusively controls the memory threshold. (PL-47746) -
OPA policy enforcement has been introduced to three new entities: Service Accounts, API Keys, and Tokens. For Service Accounts and API Keys, naming convention policies are enforced, while for Tokens, Time-To-Live (TTL) policies are enforced. These enforcement mechanisms are seamlessly integrated into both create and update operations, ensuring adherence to predefined standards during the
onSave
action. (PL-46778) -
Support added to enable OPA policy for naming convention enforcement while creating or updating a service account. (PL-46777)
Fixed issues
-
Attempts to use the
harness_platform_user
resource to create or delete users resulted in an error. The message "Request failed as you have an older version of an entity, please reload the page and try again" was displayed and the Terraform state went out of sync with Harness. This issue has been fixed. (PL-39870, ZD-47107) -
Continuous Verification for Google Cloud Operations logged error for the
resourceName
field. This issue is fixed by changing the identifier in the request body fromprojectId
toresourceName
for data collection tasks as mentioned in the Google API documentation. (CDS-89441)
Version 24.03.82408
Hotfix
-
Fixed an infinite loop issue in the delegate SCM service. (PL-48043)
-
Added support for GitOps pipeline steps with Harness Code and bumped the SCM version to
d78720584
. (CODE-1572)
Version 24.02.82406
Hotfix
- Previously, during the creation of rollback data, AWS Lambda would use string values for function versions. However, it now considers the integer values of function versions. This means that if you have deployed function versions
{8,9,10}
and you are currently deploying version{11}
, the previous rollback version will be{10}
, instead of{9}
. (CDS-92300)
February 2024
Version 24.02.82404
Hotfix
- Updated the behavior of the Scale step. After the Scale step is executed, all workload pods are published as new pods, as the scale step can be used to scale pods and change traffic on the pods. (CDS-91534, ZD-54319)
Version 24.02.82402
Fixed issues
-
The retry interval for attempting to create or read secrets from HashiCorp Vault was fixed at 1 second after each failure. (PL-46595, ZD-57053)
The retry interval has now been modified to increase by a factor of 2 times the number of failures. Consequently, after the first failure, the second attempt will occur after a 2-second delay, and the third attempt will be made after a 4-second delay, enhancing the robustness of secret management operations.
-
When linking an SSO group with over 1,000 users, only 1,000 users were syncing in Harness due to a limitation with LDAP groups syncing. (PL-46492, ZD-56741)
Implemented LDAP to perform paginated queries by default for large groups, with a fallback to non-paginated calls, ensuring complete user synchronization.
-
Pipelines were failing due to errors related to the inability to acquire delegate tasks. (PL-42600, ZD-54025, ZD-54324)
The logic for calculating CPU and Memory usage has been improved, specifically for scenarios utilizing the dynamic task request handling feature in delegates, enhancing the reliability of task allocation and pipeline execution.
-
A null pointer exception was occurring for enforcement limit accounts, triggered by the introduction of the startup plan. (GTM-3247)
This issue has been resolved by implementing an appropriate error message code for enforcement limit accounts when customers reach their enforcement limits, eliminating the null pointer exception.
-
Users were unable to create custom queries as a heath source for monitored services. (CDS-91181, ZD-57562)
This issue is fixed by making the service instance field configurable for users.
Version 24.02.82309
Hotfix
- We identified and resolved a high memory and CPU utilization issue in our delegate pods, traced back to improper handling of Chronicle libraries. The fix involved ensuring the StoreTailer objects are closed after each use, significantly improving system performance and stability. (CCM-16052)
Version 24.02.82308
Hotfix
- Upgraded the SDK for the ASG swimlane. (CDS-91937)
Version 24.02.82306
Hotfix
- Added default values for minimum healthy percentage as 90 and maximum healthy percentage as 110 for the instance refresh operation that is performed during ASG Rolling deployments to prevent service downtime. (CDS-91335, ZD-57686)
Version 24.02.82304
Hotfix
- Fixed an issue in ECS Blue Green deployments where the ECS service was deleted after the first or second deployment. (CDS-91499, ZD-57892)
Version 24.02.82303
Hotfix
- Fixed an issue for GitHub connectors when Fetch Files failed because of an NPE error. (CDS-91176, ZD-57550)
Version 24.02.82302
Behavior changes
- In the blue/green stage scale down step, we used to scale down deployments, statefulsets, daemonsets, deploymentConfig and delete HPA, and PDB resources. During scale down, we updated the field
replicas
to 0. In Kubernetes, if HPA is configured it is not mandatory to define replicas. So when another deployment happens and we apply the same old deployments manifest it does not update the replicas field and it remains set to 0. This results in no deployment even though the pipeline is successful. This issue has not been resolved. Instead, we scale down only DaemonSets and delete deployment, deploymentConfig, HPA, PDB, and statefulset resources. (CDS-88999, ZD-56645)
Fixed issues
-
Addressed an issue where pod deletion didn't trim excess whitespace in namespace names, which could prevent pod cleanup. (CI-10636, ZD-54688)
-
Fixed an issue where pipelines could fail when triggered by BitBucket PRs with more than 25 commits. This error was due to an infinite loop situation that could occur when there was pagination in the BitBucket List PR Commits API payload. (CI-11220, ZD-57421)
-
Harness CI no longer stores clone tokens for public GitHub repositories as environment variables, because a token isn't needed to clone public repos. (CI-10938)
-
The error message text for the
no eligible delegates present
error now includes additional potential causes. (CI-10933, ZD-55977)
January 2024
Version 24.01.82202
Fixed issues
-
The Azure endpoints were not being set according to the Azure environment selected, which caused the Azure connectors to function properly only for the Azure public cloud but not for other Azure cloud variations such as Azure Gov, Azure China, and so on. (PL-43333, ZD-54717)
Now, the correct Azure resource manager endpoint will be chosen based on the environment selected in the connector.
-
PR status updates now send correctly when using a GitHub App in a GitHub connector with a secret (instead of plain text) for the Application ID. (CI-11025, ZD-56177)
Version 24.01.82110
Hotfix
-
You can now hide sensitive log information in the Harness UI based on regular expression patterns. (PL-46531, ZD-56849)
For more information, go to Hide log information using regex patterns.
Version 24.01.82109
Hotfix
-
Application logs were printed in TAS deployment execution logs. (CDS-89172)
Harness added a new environment variable
DISABLE_CF_APP_LOG_STREAMING
to enhance control over this behavior. Setting this variable totrue
will redact all application logs, providing users with more flexibility in managing log visibility.
Version 24.01.82108
Early access features
-
Allowlist verification for delegate registration (PL-42471)
noteCurrently, allowlist verification for delegate registration is behind the feature flag
PL_ENFORCE_DELEGATE_REGISTRATION_ALLOWLIST
. Contact Harness Support to enable the feature.Without this feature flag enabled, delegates with an immutable image type can register without allowlist verification. With this feature flag enabled, delegates with an immutable image type can register if their IP/CIDR address is included in the allowed list received by Harness Manager. The IP address/CIDR should be that of the delegate or the last proxy between the delegate and Harness Manager in the case of a proxy.
Harness Manager verifies registration requests by matching the IP address against an approved list and allows or denies registration accordingly. For more information, go to Add and manage IP allowlists.
Fixed issues
-
Intermittent errors occurred when pulling secrets from a Custom Secret Manager. (PL-43193, ZD-54236, ZD-54555, ZD-55919)
This issue has been resolved by adding a timeout (in seconds) to fetch secrets from a custom provider in the Custom Secret Manager settings. The process interrupts and fails when it takes longer than the configured timeout to fetch the secret. The default value is 20 seconds.
-
Fixed an issue where pod creation failed in Kubernetes cluster build infrastructures if the pod volume mount key exceeded 63 characters. (CI-10789, ZD-55265)
Version 23.12.82000
Fixed issues
-
For user groups provisioned from SCIM to Harness, for the corresponding user groups created in Harness, the user group
identifier
is derived from the display name of the user group in the SCIM provider. Harness replaces.
(dots) and-
(dashes) with an_
(underscore). All other special characters (#
,?
,%
, and so on) and spaces are removed. Leading digits0
through9
and$
are also removed. (PL-42535, ZD-53830, ZD-55294)All special characters except
.
,-
, and non-leading$
and digits0
through9
are removed.Example 1: For a user group in SCIM with the name
Harness.Group?Next#Gen-First
, the user group created in Harness will have theidentifier
:Harness_GroupNextGen_First
.Example 2: For a user group in SCIM with the name
123#One.$Two.$Three.123
, the user group created in Harness will have theidentifier
:One_$Two_$Three_123
.The existing behavior of
.
and-
changed to_
has been retained.The name of the corresponding user group created in Harness will retain the special symbols as present in the user group of the SCIM provider. Example: For a user group in SCIM with the name
Harness.Group?Next#Gen-First
, the user group created in Harness will have the samename
:Harness.Group?Next#Gen-First
.
Version 24.01.82005
Hotfix
- Added extra logs to capture CI pod cleanup issues for Windows. (CI-10636, ZD-54688)
Version 24.01.82002
Hotfix
-
In the HTTP step, when a MTLS server was used, the task was not assigned to a delegate. (CDS-87547, ZD-55531)
This issue has been fixed.
Version 23.12.81811
Hotfix
-
Added support for the Tanzu application service Client ID and Secret ID via env variables in the delegate. (CDS-88086)
You can now create a Tanzu connector by setting the
AS_REFRESH_TOKEN_CLIENT_ID
,TAS_REFRESH_TOKEN_CLIENT_SECRET
,ENABLE_TAS_REFRESH_TOKEN_CLIENT_ID
parameters, and providing the Refresh token. The connector will generate a Refresh token using the Client ID and Secret ID.