企业官网建设流程全解析

GE Local Operator Feature Analysis

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge

1 Feature Overview

GE Local Operator (abbreviated as GE Local operator) is a class of "local operators" built into the GE graph engine. It handles operator nodes thatdo not require actual computation on the Ascend NPU. These operators serve as skeleton nodes in the graph—handling data transfer, control flow orchestration, constant storage, and shape inference.

Unlike engines designed for actual computation tasks such as FE (Fusion Engine) and AICPU, the GE Local engine (engine nameDNN_VM_GE_LOCAL) is a "zero-computation" engine. Operators managed by this engine complete parameter calculation or memory layout planning during compilation. At runtime, they only perform lightweight data movement or reference operations without generating any device-side kernel calls.

Core Positioning

The core problem that GE Local engine solves is:How to elegantly handle a large number of non-computational nodes in a graph compilation system designed for heterogeneous accelerators?

A typical deep learning computation graph, after being converted to AscendIR by a framework adapter (such as TorchAir), contains many non-computational nodes: data entry (Data), model output (NetOutput), constants (Constant/Const), control flow (If/While/Case), shape operations (Shape/Reshape/Squeeze), and so on. These nodes should not occupy NPU computational resources, but they still need to participate in the graph compilation, scheduling, and execution processes.

The design philosophy of GE Local is to consolidate these nodes into a dedicated engine that fulfills their "placeholder" responsibilities with minimal overhead, ensuring completeness of the compilation process and correctness of the execution flow.

2 Architecture Design

The core logic of the GE Local feature concentrates in the compilation (compiler) phase. The overall architecture is as follows:

2.1 Compilation Phase

The core code for the compilation phase is located incompiler/engines/local_engine/, producing two dynamic libraries:

2.1.1 Engine Entry (GeLocalEngine)

TheGeLocalEngineclass undercompiler/engines/local_engine/engine/adopts the singleton pattern and is loaded by the GE engine manager as a dynamic library plugin. It exposes four C-style interfaces:

• Initialize: CreatesGeLocalOpsKernelInfoStoreandGeLocalGraphOptimizerinstances
• GetOpsKernelInfoStores: Registers the operator information registry to the GE framework withDNN_VM_GE_LOCAL_OP_STOREas the key
• GetGraphOptimizerObjs: Registers the graph optimizer to the GE framework
• Finalize: Releases resources

The engine is loaded during GE initialization, following GE's plugin-based engine registration protocol—each engine dynamic library exports four unified C symbols, and the GE framework loads them viadlopenand binds by symbol name.

2.1.2 Operator Information Registration (GeLocalOpsKernelInfoStore)

GeLocalOpsKernelInfoStoreis responsible for declaring to the GE framework "which operators I support". During initialization, it retrieves the list of all registered operator types fromOpFactoryand creates a defaultOpInfostructure for each operator:

• engine = "DNN_VM_GE_LOCAL": Owning engine name
• opKernelLib = "DNN_VM_GE_LOCAL_OP_STORE": Owning operator library
• computeCost = 0: Computation cost is zero, indicating the scheduler need not perform special scheduling for these operators
• flagAsync = false,flagPartial = false,isAtomic = false: Synchronous execution, does not support partial support, non-atomic operation

TheCheckSupportedmethod implementation is extremely concise—it directly searches for matches in the registered operator name table. For GE Local operators, there is no concept of "partial support"; a type match means full support.

2.1.3 Operator Factory (OpFactory)

TheOpFactoryundercompiler/engines/local_engine/ops_kernel_store/op/adopts the registration-based factory pattern. It binds operator types with creation functions at compile time through theREGISTER_OP_CREATORmacro. The factory manages two types of operator implementations:

NoOp (Null Operation Operator)

TheRun()method ofNoOpreturns success directly without performing any operation. It covers the following categories of operators:

GeDeletedOp (Operators to be Deleted)

TheRun()method ofGeDeletedOpintentionally returns FAILEDwith detailed diagnostic information. These operators (such as Identity, Shape, Size, Rank, Placeholder, and so on)should not existin a correctly compiled graph—they should be eliminated by graph optimization passes. If these operators reach the GE Local engine, it indicates a problem with the graph optimization process.

This is a carefully designed defensive approach: instead of silently skipping or throwing vague errors, it explicitly tells the user "which optimization pass should have deleted this operator, and whether that pass is currently enabled". For example, for theShapeoperator, it checks whether constant folding (OO_CONSTANT_FOLDING) is enabled and provides targeted suggestions.

2.1.4 Graph Optimizer (GeLocalGraphOptimizer)

GeLocalGraphOptimizercurrently has substantial logic only in theOptimizeOriginalGraphJudgeInsertphase, specifically handling two virtual operators:PhonyConcatandPhonySplit:

• ForPhonyConcat: SetsNOTASK(no execution task generated),NOPADDING_CONTINUOUS_INPUT(input continuous without padding),OUTPUT_REUSE_INPUT(output reuses input memory)
• ForPhonySplit: Sets similar attributes, with the difference beingNOPADDING_CONTINUOUS_OUTPUT(output continuous without padding)

These attribute settings enable PhonyConcat/PhonySplit to be recognized as "zero-copy concatenation/split" during the memory planning phase—the memory planner knows these nodes do not need independent output buffers and only need to reference at appropriate offsets in the input buffer.

2.1.5 Operator Builder (GeLocalOpsKernelBuilder)

GeLocalOpsKernelBuilderis the core working component during compilation, implementing theOpsKernelBuilderinterface and responsible for two key tasks:

CalcOpRunningParam—Calculate Operator Running Parameters

The core work of this method is calculating the memory size of each output tensor. For GE Local operators, memory calculation has some special handling:

• Data/RefData and other data nodes: UsesGetTensorMemorySizeInBytesWithAutoPaddingto calculate aligned memory size
• Constant/Const with type DT_STRING: Uses specialized string memory calculation logicGetConstantStrMemSize
• FileConstant: Directly reads preset length fromATTR_NAME_LENGTHattribute
• PhonyConcat/PartitionedCall: Performs additional 32-byte alignment (AlignOutputMemSize)
• Unknown shape nodes: Skips calculation, determined dynamically at runtime

For specific operator types, specialized offset calculation functions are also called:

• PhonyConcat:CalcPhonyConcatNodeOffset—calculates offset positions of multiple inputs in continuous memory
• PhonySplit:CalcPhonySplitNodeOffset—calculates offset positions of multiple outputs in continuous memory
• Bitcast/Flatten/ExpandDims/ReFormat/Squeeze/Unsqueeze:CalcNodeOffsetByReuseInput—marks output to reuse input memory

PhonyConcat Offset Calculation Details

CalcPhonyConcatNodeOffset(defined in theGeLocalOpsKernelBuilderCalcOpParamclass) supports offset calculation for multi-axis concatenation. It calculates the offset position of each input node in its output buffer through theconcat_dim(concatenation axis list) andN(concatenation count list) attributes.

The calculation process uses a hierarchical slice_id approach: decomposes the operator index into position indices on each axis layer by layer, then accumulates offsets from inner to outer axes. It supports negative axis indexing (automatically converted to positive), and performs strict validity checks: input shape consistency check, 32-byte alignment check, axis attribute and tensor dimension matching check, and so on.

GenerateTask—Task Generation

The logic ofGenerateTaskis relatively simple:

• For operators likeStackPopthat depend on computation, sets theDEPEND_COMPUTEattribute to indicate shape depends on computation results
• For unknown shape nodes, sets theNOTASKattribute to skip task generation
• For other nodes, creates the corresponding Op object throughOpFactoryand callsRun()

3 User Scenarios

3.1 Scenario 1: Basic Skeleton Construction of Computation Graph

Any model compiled through GE naturally uses GE Local operators. When framework adapters (TorchAir/TFA) convert models to AscendIR, they automatically insert nodes such as Data (input nodes), NetOutput (output nodes), and Constant (weight constants). These nodes are automatically assigned to the GE Local engine by the engine scheduler, without user awareness.

3.2 Scenario 2: Shape Inference and Constant Folding

In dynamic shape scenarios, operators like Shape, Rank, and Size need to compute shape information at runtime based on actual inputs. The GE Local engine executes these computations on the Host side through the Host Kernel mechanism and copies the results to the device side for use by subsequent operators.

If the user enables constant folding optimization (OO_CONSTANT_FOLDING), these shape-related operators are folded into constants during compilation and do not enter the runtime phase.

3.3 Scenario 3: Zero-Copy Memory Reuse

Shape transformation operators such as Reshape, Bitcast, Flatten, ExpandDims, Squeeze, and Unsqueeze do not change the underlying data, only the shape description. The GE Local engine marksReuseInputduring compilation throughCalcNodeOffsetByReuseInput, and at runtime directly references the input memory, achieving zero-copy.

3.4 Scenario 4: Virtual Concatenation/Splitting (PhonyConcat/PhonySplit)

PhonyConcat and PhonySplit are virtual operators used internally by GE to represent the concatenation and splitting relationships of multiple tensors in continuous memory. During the graph optimization phase, GeLocalGraphOptimizer sets theNOTASKattribute for them. During the compilation phase,CalcPhonyConcatNodeOffset/CalcPhonySplitNodeOffsetcalculates the memory offsets for each input/output. At runtime, these nodes do not execute any operations; the actual memory sharing is coordinated by the memory planner and execution framework through offset attributes.

3.5 Scenario 5: Control Flow and Data Flow

Control flow operators such as If/While/Case/For and data flow operators such as Stack/StackPush/StackPop/StackClose are handled by the GE Local engine. Control flow operators are processed through the runtime subgraph execution mechanism, and data flow operators manage cross-node data transfer through theDataFlowResourcemechanism.

4 Operator Classification Overview

NoOp Class Operators (No task generated during compilation, null operation at runtime)

GeDeletedOp Class Operators (Should not exist in normal compilation flow, error if present)

Identity, IdentityN, Shape, ShapeN, Size, Rank, Placeholder, Switch, Snapshot, ReadVariableOp, VarHandleOp, TemporaryVariable, DestroyTemporaryVariable, GatherShapes, TransShape, and so on.

5 Key Design Decisions

5.1 Separation of Responsibilities Between Compilation and Runtime

A core design of GE Local is to push as much work as possible to the compilation phase:

• Compilation Phase: Calculate output memory size (CalcOpRunningParam), set memory reuse markers (ReuseInput), calculate PhonyConcat/PhonySplit offsets, setNOTASKattributes
• Runtime Phase: Only perform lightweight operations—reference setting, constant value output, Host shape calculation, and so on

This design enables the compilation phase to complete the vast majority of work, making the runtime execution path extremely short with negligible impact on overall inference performance.

5.2 Defensive Design of GeDeletedOp

Explicitly registering "operators that should be optimized away" asGeDeletedOpand returning an error at runtime is a strongly constrained design choice. An alternative approach could be silent skipping (like NoOp), but this would mask graph optimization issues. The current implementation exposes compilation flow anomalies at the earliest opportunity and helps users identify problems by associating optimization option names.

5.3 Zero-Copy Strategy for PhonyConcat/PhonySplit

The design of PhonyConcat/PhonySplit embodies the philosophy of "plan at compilation, zero overhead at runtime". By calculating all participants' memory offsets during compilation, these nodes execute nothing at runtime. Actual memory continuity is guaranteed by the memory planner based onCONTINUOUS_INPUT/OUTPUTand offset attributes.

6 Key Files Involved

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge