Original Publishing Date: 2022-07-18 Symptoms Some Service Plans are missing from Product Inventory. Cause Product Inventory misconfiguration. A particular resource has a different cost (recurring fee) for different Service Plans which is controverted with the official documentation https://docs.cloudblue.com/cbc/20.5/premium/content/Billing-Providers-Guide/add-products-to-product-inventory.htm The resource rate price is the same in all service plans offered by the account.  PlanID | PlanRateID |  setupFee  | recurringFee | costForAdditional --------+------------+------------+--------------+-------------------  248729 |     931171 | 0.00000000 |  48.00000000 |        0.00000000   68616 |     555114 | 0.00000000 |   4.00000000 |        0.00000000 (2 rows) Resolution There are two ways to fix the issue: 1. Unpublish one of the problem service plans. Prices are validated only in published plans. After that, Product Inventory will work correctly with the published plan. 2. Set equal prices in all service plans for the same products. After that, the product will pass prices validation. Internal Internal article

Original Publishing Date: 2022-07-27 Symptoms When upgrading your CloudBlue Commerce installation from version 20.5 to version 21.x, oss-init-job Pods fail with the "Wrong number of applications to upgrade" error. For example: # kubectl -n cbc get pods | grep oss oss-init-job-2gmtw 0/1 Error 0 8m41s ... oss-init-job-xm7dv 0/1 Error 0 14m oss-node-0                                                      1/1     Running   0          17m # kubectl -n cbc logs oss-init-job-xm7dv ... 2022-07-23 13:12:07.307 [INFO] Installing system APS packages ... ... 2022-07-23 13:12:07.491 [DEBUG] SQL: SELECT ap.version, ap.release FROM aps_package ap JOIN aps_application aa ON aa.package_id = ap.id WHERE ap.name='User and Account Management' (()) 2022-07-23 13:12:07.492 [INFO] Upgrading system APS package: "User and Account Management" from 20.4.0-144 to 21.6.0-144 2022-07-23 13:12:07.492 [DEBUG] call method pem.APS.upgradeApplication({'package_url': 'file:///build/packages/provide/uam.application-21.6.0-144.app.zip'}) 2022-07-23 13:12:21.190 [DEBUG] return {'status': -1, 'error_message': 'Wrong number of applications to upgrade', 'extype_id': -1, 'module_id': '', 'error_code': -1, 'properties': {'package_url': 'file:///build/packages/provide/uam.application-21.6.0-144.app.zip'}} 2022-07-23 13:12:21.190 [DEBUG] Traceback saved: (, OpenAPIError('Wrong number of applications to upgrade',), ) Traceback (most recent call last): Traceback (most recent call last): File "", line 1, in File "post_install.py", line 38, in main install_system_packages() File "post_install.py", line 38, in main File "post_install.py", line 29, in install_system_packages install_system_packages() post_upgrade.upgrade_system_packages() File "post_install.py", line 29, in install_system_packages File "post_upgrade.py", line 139, in upgrade_system_packages _upgrade_package(pkg, oss_pkgs[pkg]) File "post_upgrade.py", line 56, in _upgrade_package post_upgrade.upgrade_system_packages() open_api.pem.APS.upgradeApplication(package_url='file://' + pkg) File "post_upgrade.py", line 139, in upgrade_system_packages File "/usr/lib/python2.7/site-packages/poaupdater/openapi.py", line 239, in __call__ raise OpenAPIError(resp) OpenAPIError: Wrong number of applications to upgrade _upgrade_package(pkg, oss_pkgs[pkg]) File "post_upgrade.py", line 56, in _upgrade_package open_api.pem.APS.upgradeApplication(package_url='file://' + pkg) File "/usr/lib/python2.7/site-packages/poaupdater/openapi.py", line 239, in __call__ raise OpenAPIError(resp) poaupdater.openapi.OpenAPIError: Wrong number of applications to upgrade Cause There are two application instances of the same system APS application, which causes the upgrade script in those Pods to fail. For example: oss=# SELECT * FROM aps_application_instances WHERE app_id IN (SELECT app_id FROM aps_applications WHERE name='UAM'); app_instance_id | app_id | app_ver | real_app_ver | app_conf_id | uri | prov_type | master_app_instance_id | owner_id | resource_id | status | uuid -----------------+--------+------------+--------------+-------------+-------------------------------------+-----------+------------------------+----------+-------------+--------+-------------------------------------- 10237 | 474 | 20.4.0-144 | | | [https://127.0.0.1:8081/pa/rest/fake] | e | | 1 | | n | 599c5af2-0029-40aa-aa59-e15124b5511b 3633 | 474 | 20.5.0-144 | | 261 | [https://127.0.0.1:8081/pa/rest/fake] | e | | 1 | | n | e7258b20-0109-4f2d-96f5-fc4b117dac3a (2 rows) In this example, UAM is the name of the system APS application that has two application instances. That name was obtaned based on the log records above: 2022-07-23 13:12:07.492 [DEBUG] call method pem.APS.upgradeApplication({'package_url': 'file:///build/packages/provide/uam.application-21.6.0-144.app.zip'}) 2022-07-23 13:12:21.190 [DEBUG] return {'status': -1, 'error_message': 'Wrong number of applications to upgrade', 'extype_id': -1, 'module_id': '', 'error_code': -1, 'properties': {'package_url': 'file:///build/packages/provide/uam.application-21.6.0-144.app.zip'}} Resolution Remove the application instance with the least version: Enter the oss-node-0 Pod: kubectl -n CBC_NAMESPACE exec -ti oss-node-0 bash In that Pod, run the following Python script: from poaupdater import openapi api = openapi.OpenAPI() api.pem.APS.unproviderApplicationInstance(application_instance_id=APP_INSTANCE_ID_OF_APP_INSTANCE_WITH_LEAST_VERSION) Uninstall the oss component: helm -n CBC_NAMESPACE delete oss Run helmfile-deploy again: helm -n CBC_NAMESPACE upgrade helmfile-deploy a8n/helmfile-deploy --set configYaml=$(cat config.yaml | base64 -w0) --set namespace=CBC_NAMESPACE --set dockerrepo=odindevops-a8n-docker-21-public.jfrog.io

Original Publishing Date: 2022-07-25 Symptom If the "transfer domain" option is chosen while purchasing a G-Suite subscription, the following error appears in the UX1 Customer Control Panel on the Sales Order completion: Action required error on generate Google subscription with token Forbidden  The below error can be seen in /var/log/httpd/error_log in G-Suite endpoint: APS-apsgoogle[10489]:  DEBUG  10489 |C:1160328|S:1396120|googlecoreservice::_createSubscriptionWithToken## Data used for transfer the domain domain.com in the batch request Google_Service_Reseller_Subscription Object (     [customerId] => Cxxxx9     [purchaseOrderId] => 1xxxx8     [resourceUiUrl] =>     [skuId] => 1xxxxxx7     [status] =>     [subscriptionId] =>     [plan] => Google_Service_Reseller_SubscriptionPlan Object ... APS-apsgoogle[10489]:  ERROR  10489 |C:1160328|S:1396120|googlecoreservice::_getGoogleExceptionError## Error:  {   "error": {     "code": 403,     "message": "Forbidden",     "errors": [       {         "message": "Forbidden",         "domain": "global",         "reason": "forbidden"       }     ]   } } APS-apsgoogle[10489]:  ERROR  10489 |C:1160328|S:1396120|googlecoreservice::_createSubscriptionWithToken## Error transferring 1010020027. Error message: Forbidden APS-apsgoogle[10489]:  ERROR  10489 |C:1160328|S:1396120|googlecoreservice::_createSubscriptionWithToken## Error on Create Google Subscriptions: Forbidden APS-apsgoogle[10489]:  ERROR  10489 |C:1160328|S:1396120|googlecoreservice::_createSubscriptionWithToken## Error on generate Google  subscription with Token: Forbidden APS-apsgoogle[10489]:  ERROR  10489 |C:1160328|S:1396120|googlecoreservice::_getGoogleExceptionError## Error:  Error on generate Google  subscription with Token: Forbidden APS-apsgoogle[10489]: ERROR 10489 |C:1160328|S:1396120|googleaccount::postOperationAsync## Error during post Operation tranferDomain: Error on generate Google subscription with Token: Forbidden Cause The transfer key specified by the customer is not valid or expired. Resolution In order to fix the issue, a correct transfer key must be provided in the domain transfer screen. Customer has to generate a new transfer key on Google side and revoke/delete the existing one. From UX1 > Google, specify the newly generated token key to complete the provisioning. For more details, refer to the APS application guide attached.

Original Publishing Date: 2022-07-29 Symptoms Cancellation orders were stuck in "Checking provisioning conditions" status.  Cause The CF order type in the provided order flow has status RB, from which there is no transitions at all. But in accordance with design the transition from status PF to status RB should exist. The upgrade script added such transition. But there is no transitions from status RB here. Resolution In order to solve this problem there are 2 ways: 1. For this custom order flow for CF order type change transition PF->RB to PF->PR in database. And move CF order from status RB to status PR. Then this order should be cancelled automatically. 2. Configure CF order flow for statuses RB, AD, CT, TW, TP, TS, TF just like in the default order flow. This way is more correct. In order to add new statuses and their transitions, complete the following actions: 1. Log in to PCP > Billing > Settings > Order Processing > All Order Flows and click the required order flow and then the required order type. i.e "Order Flow #2000001" and "Order Type CF" 2. In the Order Statuses tab, add new statuses. Flags for them can be taken from the status before which new statuses are added, or from the same statuses from the default order flow. For Order Statuses : Order Status - RB Description - Checking provisioning conditions Order is visible to - Vendor and Provider Is Editable - Yes Is Addable - Yes Is Deletable - Yes Is Payable - Yes Is included in Order Balance - Yes Prohibit Order Placing - All Order Types (Per Subscription) Order Status - AD Description - Applying Credit Documents to Reseller-level Transactions Vendor Warning - @@LastOrderProcessingError@ Order is visible to - Vendor and Provider Is Editable - Yes Is Addable - Yes Is Deletable - Yes Is Payable - Yes Is included in Order Balance - Yes Prohibit Order Placing - All Order Types (Per Subscription) Order Status - CT Description - Automatic Payment Creation for Reseller Vendor Warning - @@LastOrderProcessingError@ Order is visible to - Vendor and Provider Is Editable - Yes Is Addable - Yes Is Deletable - Yes Is Payable - Yes Is included in Order Balance - Yes Prohibit Order Placing - All Order Types (Per Subscription) Order Status - TW Description - Automatic Payment for Reseller is Waiting for Completion Vendor Warning - @@LastOrderProcessingError@ Order is visible to - Vendor and Provider Is Editable - Yes Is Addable - Yes Is Deletable - Yes Is Payable - Yes Is included in Order Balance - Yes Prohibit Order Placing - All Order Types (Per Subscription) Order Status - TP Description - Reseller Balance Reservation in External System: Waiting for Asynchronous Response Vendor Warning - @@LastOrderProcessingError@ Order is visible to - Vendor and Provider Is Editable - Yes Is Addable - Yes Is Deletable - Yes Is Payable - Yes Is included in Order Balance - Yes Prohibit Order Placing - All Order Types (Per Subscription) Customer can cancel Order - Yes Order Status - TS Description - Reseller Balance Reservation in External System: Processing Response Vendor Warning - @@LastOrderProcessingError@ Order is visible to - Vendor and Provider Is Editable - Yes Is Addable - Yes Is Deletable - Yes Is Payable - Yes Is included in Order Balance - Yes Prohibit Order Placing - All Order Types (Per Subscription) Customer can cancel Order - Yes Order Status - TF Description - Reseller Balance Reservation in External System: Rejected Vendor Warning - @@LastOrderProcessingError@ Order is visible to - Vendor and Provider Is Editable - Yes Is Addable - Yes Is Deletable - Yes Is Payable - Yes Is included in Order Balance - Yes Prohibit Order Placing - All Order Types (Per Subscription) Customer can cancel Order - Yes 3. In the Order Flow Transitions tab, add their transitions. The new statuses must have the following transitions. Order Flow Transition 'AD' -> 'PR' -> 'CT' : From - AD Success - PR Fail - CT Order Flow Handler - Empty/Null Transition Type - On Event Action - Apply Credit Documents Order Flow Transition 'CT' -> 'TW' -> 'TS' : From - CT Success - TW Fail - TS Order Flow Handler - BM OF_CreateTopUpPayments Transition Type - Auto Action - Create Auto-Payments Order Flow Transition 'RB' -> 'AD' -> 'RB' : From - RB Success - AD Fail - RB Order Flow Handler - BM OF_CreateResellerTransactions Transition Type - On Event Action - Create Reseller-level Transactions Order Flow Transition 'TF' -> 'CL' -> 'TF' : From - TF Success - CL Fail - TF Order Flow Handler - BM OF_CancelOrder Transition Type - Manual Action - Cancel Order Order Flow Transition 'TF' -> 'CT' -> 'TF' : From - TF Success - CT Fail - TF Order Flow Handler - Empty/Null  Transition Type - Manual  Action - Check Balance and Create Auto-Payment for Reseller Order Flow Transition 'TP' -> 'CL' -> 'TP' : From - TP Success - CL Fail - TP Order Flow Handler - BM OF_CancelOrder Transition Type - Manual Action - Cancel Order Order Flow Transition 'TP' -> 'CT' -> 'TP' : From - TP Success - CT Fail - TP Order Flow Handler - Empty/Null Transition Type - Manual Action - Check Balance and Create Auto-Payment for Reseller Order Flow Transition 'TS' -> 'TP' -> 'TF' : From - TS Success - TP Fail - TF Order Flow Handler - BM OF_IsBalanceReservationInProgress Transition Type - Auto Action - Check Reseller Balance Reservation Status Order Flow Transition 'TW' -> 'AD' -> 'TW' : From - TW Success - AD Fail - TW Order Flow Handler - BM OF_WaitForPaymentsCompletion Transition Type - On Event Action - Check Auto-Payment Completion Re-submit provisioning for hanging orders.

Original Publishing Date: 2022-08-04 Pre-Prerequisites Fixed Issues Installation Procedure Help Resources Pre-Requisites This hotfix requires: Odin Automation Premium 7.4, or later, CloudBlue Commerce 20.4, 20.5, or 21.0. Fixed Issues CCU-21251 The problem with the related backup schedule for postponed backups has been fixed. CCU-21263 The problem with the Plesk version has been fixed. Configuration Plesk configuration in the configuration file on the OACI endpoint node /usr/local/share/PACI-aps/paci-config.xml ... https://10.26.245.75:6308 PANEL_18_0_42_WIN PLESK_18_0_42 600 3600 7200 60 180 1800 1200 360 3600 600 360 30 300 1800 1200 Back-upconfiguration in the configuration file on the IM node /us/local/share/PACI-im/IM-config.xml Note: Check the configuration operability in a sandbox before using on the production environment. Installation Procedure To install the Cloud Infrastructure Automation application, follow the instructions provided in the documentation.  Help Resources Cloud Infrastructure Integration Guides CloudBlue Commerce platform documentation Virtuozzo documentation Plesk documentation Obtaining To obtain the Cloud Infrastructure 18.0 hotfix 36, refer to this article.

Original Publishing Date: 2022-08-08 Symptoms Prechecks that you perform for your CloudBlue Commerce databases with the db_precheck.py script cannot be successfully completed because there are failed tasks "Periodic check for certificates statuses" in BSS. For example: ID 299373875 Event 299595044 Event Handler Periodic check for certificates statuses CERTGATE :: CheckCertStatusesTask Status Failed Start Time 16-Jul-2022 20:51:00 End Time 01-Jan-1970 01:00:00 Redo Counter 1 Comments Execution Failed: container CERTGATE is not available. Transaction ID 0 Problem Report 4504795 Priority Normal Sticky Priority No These tasks appear again after their cancellation. Resolution Set the next generation of the "Periodic check for certificates statuses" event on a date in the future so that you will have enough time to perform the upgrade: In the provider control panel (BSS), go to System > Settings > Events. Select the "Periodic check for certificates statuses" event. Select the Scheduler tab. Reconfigure the schedule so that you will have enough time to perform the upgrade. Remove all failed tasks "Periodic check for certificates statuses": In the provider control panel (BSS), go to Operations > Tasks. Select the Failed Tasks tab. Remove all failed tasks "Periodic check for certificates statuses". Create new dumps of your CloudBlue Commerce databases. Perform the database prechecks again.

Original Publishing Date: 2022-08-09 Symptoms Upon placement of order in UX1, the error "Unable to complete the operation, please cancel and try again later" will be prompted to the user. The error we can observe via browser inspect can be seen as per below : {"fullMessage":"com.ingrammicro.imcp.library.aps.exception.APSInvocationException: Incorrect Response from APS Controller, Status Code is '500' caused by: com.parallels.pa.rest.BillingAPIFault: Plug-in is not configured. ","cause":{"error":"com.parallels.bss.rest.errors.BSSAPSError","packageId":"56d95e6b-4d23-4b0b-a0b5-7929c6029b22","message":"com.parallels.pa.rest.BillingAPIFault: Plug-in is not configured. ","http_request":"POST https://127.0.0.1:8081/pa/billing/9d09b1c5-ccf0-4214-af00-8f3c17b30906/om/xmlrpc/calc-sales-order-api"}},"message":"Plug-in is not configured. ","http_request":"POST https://ordermanagement:8081/rest/order-management/be6aed13-3ebc-4823-bbb6-5d91c2cf8b5b/orders/estimate","pkgId":"821cb973-4335-4d62-9267-33ac860ed836"} Cause The root cause can be identified by investigating billing.log where the Exception message can be seen with identical error message - "Plug-in is not configured" :  [22-04-12 22:28:10.968 VERTEX_Obje RQ2872763 TH01957 TRC] ...[6] *** INTERRUPTED BY EXCEPTION *** ItemResult* VERTEX::CalculateTaxes(DataPacket*) [22-04-12 22:28:10.968 VERTEX_Obje RQ2872763 TH01957 NTE] Finished method VERTEX.CalculateTaxes(user = 1, SID = 0, lang = en, request = 0, localObject = 1, transaction = 4248317 (HP)) [22-04-12 22:28:10.969 VERTEX_Obje RQ2872763 TH01957 NTE] ExceptionResult: Plug-in is not configured. [22-04-12 22:28:10.969 RO_2:136 RQ2872763 TH01957 TRC] ---[5] DataPacket* RemoteObject::Call(const string&, DataPacket&, int) [22-04-12 22:28:10.969 RemoteRes RQ2872763 TH01957 NTE] -> User-defined exception #-2147483648 Plug-in is not configured. [22-04-12 22:28:10.973 RO_2:136 RQ2872763 TH01957 TRC] (RO_2:136) disconnected. [22-04-12 22:28:10.973 GenericWork RQ2872763 TH01957 TRC] ...[4] *** INTERRUPTED BY EXCEPTION *** virtual TaxUniversalCalcResult bTax::calcTaxLod(const Account&, const std::map&, Int, Int, Int, Int, Int, TaxUniversalDocDets&, TaxUniversalTaxDets&) [22-04-12 22:28:10.973 GenericWork RQ2872763 TH01957 TRC] ...[3] *** INTERRUPTED BY EXCEPTION *** virtual void bSalesOrder::applyTaxes(Int, std::list&, bool, TaxUniversalTaxDets&) [22-04-12 22:28:10.974 GenericWork RQ2872763 TH01957 TRC] ...[2] *** INTERRUPTED BY EXCEPTION *** virtual void bSalesOrder::calculateTotals(Int, SalesOrderBSS&, Str, bool, bool) [22-04-12 22:28:10.974 GenericWork RQ2872763 TH01957 TRC] ...[1] *** INTERRUPTED BY EXCEPTION *** virtual SalesOrderBSS bSalesOrder::calculateSalesOrder(const SalesOrderParamsBSS&) [22-04-12 22:28:10.974 BM_Object RQ2872763 TH01957 TRC] ...[0] *** INTERRUPTED BY EXCEPTION *** ErrorMessage* BM::CalcSalesOrder_API(Str) [22-04-12 22:28:10.974 BM_Object RQ2872763 TH01957 NTE] Finished method BM.CalcSalesOrder_API(user = 1, SID = 0, lang = en, request = 2872763, localObject = 0, transaction = 4248317 (HP)) [22-04-12 22:28:10.974 BM_Object RQ2872763 TH01957 NTE] ExceptionResult: Plug-in is not configured. The exception error can be seen to be triggered by Tax calculation for sales order and the method VERTEX::CalculateTaxes  Resolution It is required to investigate vertex settings of the customer's Reseller in Billing > Settings > Taxation > Vertex > General. If the reseller is configured to use vendor vertex settings, it is required to match the vertex configuration of Reseller's Provider inclusive of the Company Code. In example : If Reseller L2 configuration Company Code is empty while Reseller's Provider which in this case is L1 have Company Code 9999, the error "Plug-in is not configured" can be triggered. Reseller L2 L2's Provider (Which is Reseller L1) Note : It is recommended to also check L1's configuration to match Provider(L0) Internal

Original Publishing Date: 2022-09-08 Symptoms Ingram Micro copyright are shown on CP login screen after license update. Example: Cause hide_parallels_copyrights license property not synched to IDP. The issue is acknowledged as software-related issue OA-15190. Resolution Please contact your Technical Account Manager to clarify the status of the software-related issue. For a workaround, please contact CloudBlue Technical Support. Internal Link to internal article 

Original Publishing Date: 2022-09-16 Symptoms Tried to update inhouse-products to version 1.4.186 but the update failed with the following error: [root@osscore ~]# helm upgrade inhouse-products a8n/inhouse-products --version 1.4.186 Error: UPGRADE FAILED: Service "camunda-rest" is invalid: spec.ports[0].nodePort: Forbidden: may not be used when `type` is 'ClusterIP' Cause The failure is caused by the following product issue PDI-1421, which cannot be fixed in the code. Note: The failure happens only in environments with old versions of K8s/helm. Resolution Upgrade helm/K8s. Example configuration where the issue does not occur: [root@mn7 ~]# kubectl version Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:31:21Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:04:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"} [root@mn7 ~]# helm version version.BuildInfo{Version:"v3.9.0", GitCommit:"7ceeda6c585217a19a1131663d8cd1f7d641b2a7", GitTreeState:"clean", GoVersion:"go1.17.5"} Alternatively you may apply the below workaround. Workaround 1. Backup the objects before deleting them: kubectl get svc camunda-rest -o yaml > camunda-rest.yaml kubectl get cm inhouse-products-logging-config -o yaml > inhouse-products-logging-config.yaml 2. Delete the objects causing the conflict: kubectl delete svc camunda-rest kubectl delete cm inhouse-products-logging-config 3. Re-try inhouse-products upgrade. It should complete successfully.

Original Publishing Date: 2022-09-20 As CloudBlue Connect continues to be the platform that powers ISV integration, we have made the strategic decision to End-of-Life (EOL) all released versions of the Amazon Web Services (AWS) APS based on the Application Packaging Standard (APS). The EOL will take effect as of December 1, 2022. At that time, the AWS APS will no longer undergo further development or maintenance, and CloudBlue will no longer accept support requests. We invite all partners to upgrade to one of our Connect-based integration options, as described in our Connector Documentation Portal. Please reach out to your Account Management team if you require any assistance or if you would like to schedule a demo.