#Install InferNex Bridge

This guide covers Alauda Build of InferNex Bridge v26.6.0, published in the OLM alpha channel for arm64 clusters.

#Prerequisites

Before installing Alauda Build of InferNex Bridge, ensure the target cluster has the required platform and inference dependencies.

#Required Dependencies

#CRDs Installed by This Operator

The Alauda Build of InferNex Bridge OLM bundle installs the two product APIs and three CRDs used by its managed elastic orchestration components:

The following CRDs are not installed by this OLM bundle. Install them separately according to the requirement shown below:

The rolebasedgroups.workloads.x-k8s.io CRD is provided by the upstream RoleBasedGroup (RBG) project. It is a conditional dependency and is not required for Deployment or LeaderWorkerSet targets. To install the upstream RBG v0.7.0 controller and CRDs, use the versioned release manifest:

kubectl apply --server-side -f \
  https://raw.githubusercontent.com/sgl-project/rbg/v0.7.0/deploy/kubectl/manifests.yaml

kubectl wait deployment/rbgs-controller-manager \
  --namespace rbgs-system \
  --for=condition=Available \
  --timeout=5m

kubectl get crd rolebasedgroups.workloads.x-k8s.io \
  -o jsonpath='{range .spec.versions[?(@.name=="v1alpha2")]}{.name}{" served="}{.served}{" storage="}{.storage}{"\n"}{end}'

The expected result is v1alpha2 served=true storage=true. The manifest uses docker.io/rolebasedgroup/rbgs-controller:v0.7.0-888ade7, which provides an arm64 image. For a disconnected cluster, mirror this image to a registry accessible from the cluster and update the manifest image reference before applying it.

See the upstream RBG releases when selecting a different version.

Enabling the managed Tidal, ElasticScaler, or ResourceScalingGroup components deploys their controller workloads. Scaling policy resources such as Tidal and ElasticScaler must still be created separately by the user or platform workflow.

#Runtime Templates and Images

#Configure ImageWhiteList

The platform creates an operator-managed ImageWhiteList when the InferNex Bridge package is uploaded. Do not create or delete this object manually. Operator-managed ImageWhiteList objects are reconciled by olm-registry and may be recreated after deletion.

Before creating an inference service, verify that the generated object contains the required source image references and source-to-target rewrite rules. Keep unrelated entries that are already present.

The following table shows the expected repository mappings. Each rule preserves the original image tag.

The relevant part of the final object should look like this. The complete object contains additional package images and platform-managed entries.

apiVersion: app.alauda.io/v1alpha1
kind: ImageWhiteList
metadata:
  name: infernex-bridge
  namespace: cpaas-system
spec:
  repoList:
    - cr.openfuyao.cn/openfuyao/cache-indexer:26.6.0
    - hub.oepkgs.net/openfuyao/ascend/vllm-ascend:v0.18.0
    - busybox:1.36.1
    - docker.io/library/busybox:1.36.1
  rewriteRules:
    - regexp: ^cr\.openfuyao\.cn/openfuyao/cache\-indexer:(.*)$
      replacement: <target-registry>/3rdparty/openfuyao/cache-indexer:${1}
    - regexp: ^hub\.oepkgs\.net/openfuyao/ascend/vllm\-ascend:(.*)$
      replacement: <target-registry>/3rdparty/openfuyao/ascend/vllm-ascend:${1}

The vllm-ascend entry must be added explicitly because the image is not part of the operator relatedImages. Import the arm64 image into the target registry before creating inference workloads. For example:

export TARGET_REGISTRY='<target-registry-host:port>'

skopeo copy --all \
  docker://hub.oepkgs.net/openfuyao/ascend/vllm-ascend:v0.18.0 \
  "docker://${TARGET_REGISTRY}/3rdparty/openfuyao/ascend/vllm-ascend:v0.18.0"

For BusyBox, retain both busybox:1.36.1 and docker.io/library/busybox:1.36.1 in repoList, but do not add a custom BusyBox rewrite rule. The platform _default rule handles the short name and Docker Hub normalization.

You can back up and edit the operator-managed object with these commands. If the object has a different name, locate it with kubectl get imagewhitelist -A.

kubectl get imagewhitelist -n cpaas-system infernex-bridge -o yaml \
  > imagewhitelist-infernex-bridge-backup.yaml

kubectl edit imagewhitelist -n cpaas-system infernex-bridge

The workload specification may continue to show the community source image. The rewrite is successful only when the actual Pod imageID points to the target registry. A package-owned component and the package-external runtime should resolve as follows:

cache-indexer=<target-registry>/3rdparty/openfuyao/cache-indexer@sha256:...
main=<target-registry>/3rdparty/openfuyao/ascend/vllm-ascend@sha256:...

Verify the imported image architecture and inspect the workload image IDs:

export TARGET_REGISTRY='<target-registry-host:port>'
export INFERENCE_NAMESPACE='<inference-service-namespace>'

crane config --platform=linux/arm64 \
  "${TARGET_REGISTRY}/3rdparty/openfuyao/ascend/vllm-ascend:v0.18.0" |
  python3 -c 'import json, sys; data=json.load(sys.stdin); assert data.get("architecture") == "arm64", data; print("arm64 image verified")'

kubectl get pods -n "${INFERENCE_NAMESPACE}" \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{range .status.initContainerStatuses[*]}  init:{.name}{"="}{.imageID}{"\n"}{end}{range .status.containerStatuses[*]}  {.name}{"="}{.imageID}{"\n"}{end}{end}'

#Prepare the Model

Model artifacts are not included in the InferNex Bridge operator package. Use the Alauda AI model catalog to prepare a ModelCar when OCI model storage is selected. See Model Storage.

#Optional Dependencies

#Optional EagleEye Observability

EagleEye is disabled by default in the Alauda OLM templates. Disabling EagleEye does not affect the inference engine, aggregate or prefill-decode deployment modes, Mooncake, cache-indexer, or PD-Orchestrator. It disables only the following optional observability workloads:

Before enabling EagleEye, verify that NATS has a ready Service and endpoint:

kubectl get service -n nats nats
kubectl get endpointslice -n nats -l kubernetes.io/service-name=nats

The EagleEye init containers connect to nats.nats:4222. If this endpoint is unavailable, the enabled EagleEye Pods remain in the init phase.

Enable EagleEye explicitly in an InferNexService or a referenced InferNexServiceConfig:

spec:
  components:
    eagleEye:
      hardwareMonitor:
        enabled: true
      hardwareDiagnosis:
        enabled: true
      networkPerformanceExporter:
        enabled: true

After applying the resource, verify the optional workloads in the inference service namespace:

kubectl get deployment,daemonset,service -n <inference-service-namespace> \
  -l infernex.io/owner=<infernex-service-name>

Expected workloads include the EagleEye hardware diagnosis Deployment, hardware monitor DaemonSet, and network performance exporter DaemonSet. If EagleEye is not required, keep all three child switches set to false.

#Upload Operator

Obtain the arm64 installation package from your Alauda product delivery channel: infernex-bridge.alpha.arm64.v26.6.0.tgz.

Use the violet command to publish it to the platform repository:

violet push --platform-address=<platform-access-address> --platform-username=<platform-admin> --platform-password=<platform-admin-password> infernex-bridge.alpha.arm64.v26.6.0.tgz

#Install Operator

In Administrator view:

1. Click Marketplace / OperatorHub.
2. At the top of the console, from the Cluster dropdown list, select the destination cluster where you want to install the InferNex Bridge Operator.
3. Search for and select Alauda Build of InferNex Bridge, then click Install.
4. Select the alpha Channel.
5. Select version v26.6.0.
6. Leave Installation Location unchanged, it should be infernex-system by default.
7. Select Manual for Upgrade Strategy.
8. Click Install.

#Verification

Confirm that the Alauda Build of InferNex Bridge tile shows one of the following states:

• Installing: installation is in progress; wait for this to change to Installed.
• Installed: installation is complete.

Verify that the operator controller and webhooks are running:

kubectl get pods -n infernex-system
kubectl get crd \
  infernexservices.infernex.infernex.io \
  infernexserviceconfigs.infernex.infernex.io \
  resourcescalinggroups.autoscaling.openfuyao.com \
  elasticscalers.elasticscaler.io \
  tidals.tidal.io

kubectl get mutatingwebhookconfiguration -o json |
  python3 -c 'import json, sys; data=json.load(sys.stdin); versions={version for item in data["items"] for webhook in item.get("webhooks", []) for rule in webhook.get("rules", []) if "llminferenceservices" in rule.get("resources", []) for version in rule.get("apiVersions", [])}; assert "v1alpha2" in versions, versions; print("LLMInferenceService webhook versions:", sorted(versions))'

The controller pod should be Running, all five bundled CRDs should be established, and the LLMInferenceService mutating webhook should include v1alpha2.

#Community Examples

For community-maintained examples, see InferNex Bridge v26.6.0 examples.

#Upgrading Alauda Build of InferNex Bridge

1. Back up the existing custom resources:

   kubectl get infernexservice,infernexserviceconfig -A -o yaml > infernex-bridge-backup.yaml

2. Upload the new version of the Alauda Build of InferNex Bridge operator package using the violet tool.

3. Go to the Administrator -> Marketplace -> OperatorHub page, find Alauda Build of InferNex Bridge, review the target version, and click Confirm to apply it.

#Verification

After upgrading, confirm that the Alauda Build of InferNex Bridge tile shows Installed and verify the controller and CRD status:

kubectl get pods -n infernex-system
kubectl get crd \
  infernexservices.infernex.infernex.io \
  infernexserviceconfigs.infernex.infernex.io \
  resourcescalinggroups.autoscaling.openfuyao.com \
  elasticscalers.elasticscaler.io \
  tidals.tidal.io

#Rollback

Use a previous package only when its CRDs are compatible with the custom resources already stored in the cluster. In OperatorHub, select the previous available version and confirm the manual upgrade action. If the previous version is not available, upload its installation package with violet first.

After rollback, verify the controller, CRDs, inference services, runtime image IDs, and one representative inference request. Do not delete the five bundled CRDs while custom resources still exist; deleting them also deletes the stored custom resources.