[WIP][TrimmableTypeMap] Decompose _GenerateJavaStubs: shared tasks for trimmable path#10810
Draft
simonrozsival wants to merge 50 commits intodev/simonrozsival/trimmable-typemap-03-generatorsfrom
Conversation
83a50be to
4e7d06c
Compare
1459e93 to
01136d4
Compare
Add the TrimmableTypeMap scanner project (netstandard2.0) with: - JavaPeerInfo, MarshalMethodInfo, ActivationCtorInfo records — the core data model representing Java peer types discovered in assemblies - SignatureTypeProvider — decodes method signatures from metadata to extract parameter types for marshal method and activation constructor matching - CustomAttributeTypeProvider — decodes custom attribute arguments with lazy enum type caching and correct nested type resolution - CompilerFeaturePolyfills — netstandard2.0 shims for required/init - NullableExtensions — IsNullOrEmpty/IsNullOrWhiteSpace helpers - System.Reflection.Metadata 11.0.0-preview.1 package reference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add AssemblyIndex — the first phase of the scanner pipeline that reads a single assembly and indexes all Java peer metadata: - Discovers [Register], [Export] attributes on types and methods - Builds RegisterInfo/ExportInfo records from custom attribute blobs - Resolves TypeAttributeInfo for component attributes ([Activity], [Service], [BroadcastReceiver], [ContentProvider]) including their JNI name properties - Maps type definitions to their Java peer registration data for downstream consumption by JavaPeerScanner Key design: uses System.Reflection.Metadata directly (no Cecil) and produces immutable record types. The index is per-assembly so scanning can be parallelized across the app closure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add JavaPeerScanner — the second phase that consumes AssemblyIndex results and produces the final list of JavaPeerInfo for the app: - Walks all indexed types and resolves their Java peer registrations - Handles inheritance: traverses base types across assemblies to find the nearest registered Java peer ancestor - Detects activation constructors (IntPtr+JniHandleOwnership) and distinguishes between direct declarations and inherited ones - Collects marshal methods ([Register] on methods) and exported methods ([Export]) with their JNI signatures - Merges component attribute metadata ([Activity], etc.) and resolves JNI names from attribute properties - Flags types for unconditional preservation when component attributes with non-default JNI names are present The scanner is designed as a pure function: assemblies in → peer info out, with no side effects or global state. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add test infrastructure for the Java peer scanner: - TestFixtures project with stub Mono.Android attributes ([Register], [Activity], [Service], etc.) and test types covering MCW bindings, user types, generics, nested types, interfaces, and component types - JavaPeerScannerTests with test helpers (ScanFixtures, FindByJavaName) and foundational assertions: type discovery, DoNotGenerateAcw flags, component unconditional marking, interface/abstract/generic metadata Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Test marshal method collection, JNI signature decoding, activation constructor resolution, base type chain walking, interface resolution, compat JNI names, and component attribute metadata merging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover generic base/interface type specification resolution, component- only base detection, unregistered nested type naming, deep nesting, empty namespace types, plain subclass CRC64 naming, and unregistered types with interfaces or [Export] methods. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add TrimmableTypeMap, TrimmableTypeMap.Tests, and TestFixtures projects to Xamarin.Android.sln. Add CI step to run scanner unit tests and publish results on the Windows build pipeline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add IntegrationTests and UserTypesFixture projects to Xamarin.Android.sln. Add CI step to run integration tests and publish results. Add InternalsVisibleTo for the integration test assembly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implements generators for #10799: - JcwJavaSourceGenerator: generates .java files for ACW types from JavaPeerInfo - TypeMapModelBuilder: transforms JavaPeerInfo → TypeMapAssemblyModel (IR/AST) - TypeMapAssemblyEmitter: mechanical SRM-based PE emitter from IR model - TypeMapAssemblyGenerator: high-level API composing builder + emitter - JniSignatureHelper: JNI signature parsing and CLR type encoding Key design: - 1 typemap assembly per input assembly for better caching - IR model separates 'what to generate' from 'how to serialize to IL' - Model builder tests are the primary unit tests (148 total, all passing) Generated assemblies contain: - [assembly: TypeMap] attributes per JNI type - Proxy types (JavaPeerProxy subclasses) with CreateInstance, TargetType - [UnmanagedCallersOnly] UCO wrappers for marshal methods/constructors - RegisterNatives with function pointer registration - IgnoresAccessChecksToAttribute for cross-assembly calls
- TypeMapAssemblyModel → TypeMapAssemblyData - TypeMapEntryModel → TypeMapAttributeData - ProxyTypeModel → JavaPeerProxyData - TypeRefModel → TypeRefData - UcoMethodModel → UcoMethodData - UcoConstructorModel → UcoConstructorData - NativeRegistrationModel → NativeRegistrationData - TypeMapModelBuilder → ModelBuilder
30 new tests covering every test fixture type: - MCW types: Object, Activity, Throwable, Exception, Service, Context, View, Button - User ACW types: MainActivity, MyHelper, TouchHandler, CustomView, AbstractBase - Interface types: IOnClickListener (with invoker dedup) - Nested types: Outer$Inner, ICallback$Result proxy naming - Multi-interface: ClickableView, MultiInterfaceView - Export methods: ExportExample - Generic types: GenericHolder - Full pipeline tests: scan → model → emit → read back PE Validates UCO wrapper signatures for all JNI types (bool, int, float, long, double, object, array), constructor wrappers, native registrations, TypeMap attribute counts, and proxy type names in emitted assemblies. 178 tests pass, 1 skipped.
…erator
Critical changes for TrimmableTypeMap:
- TypeMapAttributeData now supports 2-arg (unconditional) and 3-arg (trimmable):
- 2-arg: ACW user types (Android can instantiate), essential runtime types
(java/lang/Object, Throwable, Exception, etc.)
- 3-arg: MCW bindings and interfaces — trimmer preserves proxy only if
target type is referenced by the app
- Alias detection: when multiple .NET types share the same JNI name,
they get indexed entries ("jni/name", "jni/name[1]", "jni/name[2]")
with distinct proxy types
- RootTypeMapAssemblyGenerator: generates _Microsoft.Android.TypeMaps.dll
with [assembly: TypeMapAssemblyTarget("name")] for each per-assembly
typemap assembly
208 tests pass, 1 skipped.
…overage Review-driven fixes: Bug fixes: - Fix critical bug: constructor JNI signatures were hardcoded to "()V" in BuildNativeRegistrations. Now propagated from JavaConstructorInfo through UcoConstructorData.JniSignature to NativeRegistrationData. - Fix non-deterministic alias ordering: replaced Dictionary with SortedDictionary, sort alias peers by ManagedTypeName. - Fix Export attribute ThrownNames parsing: string[] was not being decoded from ImmutableArray<CustomAttributeTypedArgument<string>>. Test improvements: - Cache scanner results with static Lazy<> in all test classes - Add parameterized constructor JNI signature test - Add fixture-based CustomView constructor signature assertions - Add PE blob validation tests (2-arg vs 3-arg TypeMap attributes) - Add determinism test (same input → same output) - Add Export with throws clause test fixture + JCW test - Fix Build_CreatesOneEntryPerPeer for alphabetical ordering Code quality: - Add ECMA-335 comment explaining Type args as serialized strings - Add comment explaining UCO constructor 2-param signature - Fix doc comment: remove 'Intermediate representation' wording 215 tests pass, 1 skipped.
…uted IgnoresAccessChecksTo, edge-case tests
…exclusion, invoker filtering, managed-name proxy naming, root assembly cleanup - Make ModelBuilder a static class with Build() method - Add IsInvokerType() to filter invokers from alias grouping (get proxy, no TypeMap entry) - Add IsImplementorOrEventDispatcher() to exclude from unconditional entries - Change proxy naming from JNI-based to managed-name-based for uniqueness - Remove unused stringRef/systemRuntimeInteropServicesRef from RootTypeMapAssemblyGenerator - Add Implementor and EventDispatcher test fixtures - Add tests for Implementor/EventDispatcher trimmability - Add root assembly attribute blob value verification test - All 225 tests pass
…ame heuristics, add tests - Clear _typeRefCache in Emit() alongside _asmRefCache to prevent stale handles on reuse - Extract MonoAndroidPublicKeyToken and SystemRuntimeVersion as named constants - Document name-based IsInvokerType/IsImplementorOrEventDispatcher heuristic limitations - Fix stale doc comment on JavaPeerProxyData.TypeName - Add FullPipeline_Mixed2ArgAnd3Arg_BothSurviveRoundTrip test - Add FullPipeline_CustomView_UcoConstructorHasExactlyTwoParams PE-level test - Add FullPipeline_GenericHolder_ProducesValidAssembly PE pipeline test - Add edge case tests documenting name-based detection limitations - Refactor ReadFirstTypeMapAttributeBlob into ReadAllTypeMapAttributeBlobs - All 230 tests pass
…support - Invoker types no longer get their own proxy types or TypeMap entries. They are only referenced as a TypeRef in the interface proxy's get_InvokerType property and CreateInstance method. - Invoker detection now uses explicit relationship from [Register] third argument (InvokerTypeName) instead of name-based heuristic. - Interface proxy CreateInstance creates the invoker type, not the interface itself. - Added dotnetVersion parameter to TypeMapAssemblyEmitter, TypeMapAssemblyGenerator, and RootTypeMapAssemblyGenerator constructors (will be passed from $(DotNetTargetVersion) MSBuild property later). - JCW generator now uses SuperArgumentsString for [Export] constructor super() calls instead of always forwarding all parameters. - Documented PE-reading test helpers with clear comments explaining the approach and its limitations.
…Attribute<Java.Lang.Object> Instead of defining a non-generic TypeMapAssemblyTargetAttribute in the root assembly, reference the existing generic TypeMapAssemblyTargetAttribute`1 from System.Runtime.InteropServices and close it with Java.Lang.Object as the type argument. This matches the runtime API: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.typemapassemblytargetattribute-1
Each '// ---- Section ----' comment block is now a nested class inside the outer test class. This allows running individual test groups in isolation and makes test failures easier to locate. Shared helpers remain as static methods on the outer class.
Generate full marshal method bodies for [Export] methods and constructors instead of skipping them or using TypeManager.Activate. For [Export] methods, the UCO wrapper emits: - BeginMarshalMethod guard - GetObject<T> to retrieve the managed peer - Parameter unmarshaling (primitives pass through, strings via JNIEnv.GetString, objects via GetObject<T>) - Managed method call - Return marshaling (strings via JNIEnv.NewString, objects via JNIEnv.ToLocalJniHandle) - catch/finally with OnUserUnhandledException and EndMarshalMethod For [Export] constructors, the wrapper additionally calls ActivateInstance before GetObject<T> to create the peer, then calls the user constructor body. JCW Java output now uses nctor_N native methods for ALL constructors (both [Register] and [Export]), removing the TypeManager.Activate codepath entirely. Changes: - TypeMapAssemblyEmitter: EmitExportMarshalMethod with full IL try/catch/finally using ControlFlowBuilder - ModelBuilder: [Export] methods/ctors populate ExportMarshalMethods list with managed parameter and return type info - JcwJavaSourceGenerator: Removed WriteTypeManagerActivate, unified constructor handling - Scanner: Captures ManagedReturnType for [Export] methods - 5 new emitter-level tests verifying export assembly generation - Updated JCW and model tests for new patterns
The 'leave' IL instruction clears the evaluation stack, so non-void export methods were losing their return values. Add a local variable (local 3) to store the return value before 'leave' and load it before 'ret'. Void methods are unaffected (3 locals as before).
The managed method ref for [Export] methods with object return types incorrectly encoded the return type as IntPtr instead of the actual managed type. The CLR would fail to find the method at runtime. Also assembly-qualify the ManagedReturnType in the scanner (matching how parameters are already handled) and add a test assertion for it.
RegisterNatives must use the name matching the JCW native method
declaration. For methods, JCW declares 'n_<ManagedMethodName>' but
RegisterNatives was using 'n_<JniName>' (stripped from wrapper name).
When the export name differs from the managed name (e.g.,
[Export("attributeOverridesNames")] CompletelyDifferentName),
the JNI runtime would fail to connect them.
Fix: pass NativeCallbackName explicitly through ExportMarshalMethodData.
For methods: 'n_<ManagedMethodName>' (matches JCW).
For constructors: 'nctor_N' (matches JCW).
[Export] methods are new declarations, not overrides of base class methods. The @OverRide annotation should only appear on [Register] methods. Also change native method declarations from 'public native' to 'private native' to match legacy JCW output. Added tests: @OverRide suppression for [Export], private native visibility assertions.
Static [Export] methods: - Added IsStatic to MarshalMethodInfo (scanner) and ExportMarshalMethodData (model) - Scanner captures MethodAttributes.Static for all methods - JCW emits 'public static' wrapper and 'private static native' declaration - IL emitter: static methods skip GetObject<T>/callvirt, use direct call - Method signature uses isInstanceMethod: false for static exports [ExportField]: - Added ExportFieldInfo data type with FieldName, MethodName, IsStatic - Scanner: ExportFieldAttribute parsed, method registered as marshal method (Connector = null, like [Export]) and field info collected separately - JCW: field declarations emitted after static initializer, before ctors Format: 'public [static] <type> FIELD_NAME = MethodName ();' 17 new tests across scanner, model, emitter, and JCW layers. 299 tests pass.
- Merge CollectMarshalMethods + CollectExportFields into single-pass CollectMarshalMethodsAndExportFields - Inline TryGetMethodRegisterInfo into merged method - Replace CollectExportFields with ParseExportFieldName helper - Extract WriteThrowsClause helper in JCW generator - Deduplicate parameter unmarshal loop in emitter - Fix stale IsExport comment
Verify that [Export] methods, constructors, static exports, and [ExportField] backing methods all appear in the RegisterNatives IL body with correct JNI names and signatures. This catches bugs where wrappers are generated but not registered, which would cause runtime UnsatisfiedLinkError.
JNI's jboolean is an unsigned 8-bit type. The legacy MarshalMethodsAssemblyRewriter maps System.Boolean → System.Byte. We were using SByte (signed), which is incorrect.
All constructors (both [Register] and [Export]) now generate full marshal bodies with BeginMarshalMethod + try/catch/finally. There are no pre-existing n_* callback methods for constructors — they only exist for [Register] methods where the connector points to a real static method. Constructor callback pattern (nctor_N_uco): GetUninitializedObject(typeof(T)) -> SetHandle(native__this, DoNotTransfer) -> .ctor(params) CreateInstance for inherited activation ctor calls BaseType::.ctor(IntPtr, JniHandleOwnership) on the first base class that declares it, rather than SetHandle directly. Removed ActivateInstance, UcoConstructorData, and the forwarding approach for constructors. Scanner now populates ManagedType on constructor parameters regardless of Connector value.
- Support arrays in export marshal wrappers using JNIEnv.GetArray<T>/NewArray<T>/CopyArray<T> including copy-back cleanup for array parameters - Support enum signatures and marshaling as JNI int while calling managed enum methods - Map Java.Lang.ICharSequence to Ljava/lang/CharSequence; in export signature generation - Resolve JNI object descriptors from [Register] metadata for managed object types (e.g. View[] -> [Landroid/view/View; instead of Object[]) - Improve ManagedTypeToAssemblyQualifiedName to resolve non-BCL types and array element types - Generate proxies for export-only ACW types (no activation ctor / no [Register] methods) Add fixture coverage and tests for export-only proxies plus array/enum/charsequence signatures and registration. All 312 tests pass.
Extract all typemap-related MSBuild targets from Xamarin.Android.Common.targets and Microsoft.Android.Sdk.ILLink.targets into a dedicated Xamarin.Android.TypeMap.Legacy.targets file, imported conditionally based on _AndroidTypeMapImplementation. Create empty stub files for the trimmable typemap implementation (Trimmable.targets, Trimmable.CoreCLR.targets, Trimmable.NativeAOT.targets). This is a pure refactoring — no new tasks, no new C# code, no behavioral changes. When _AndroidTypeMapImplementation != 'trimmable' (the default), the legacy targets are imported and the build pipeline is identical to before. Changes: - New: Xamarin.Android.TypeMap.Legacy.targets — all legacy typemap targets - _GenerateLegacyJavaCallableWrappers (JCW gen, ACW map, stubs, marshal methods) - _GenerateLegacyAndroidManifest (manifest + additional providers) - _GenerateLegacyTypeMappings (LLVM IR typemaps) - _SetLegacyTypemapProperties (_TypeMapKind) - _GetMonoPlatformJarPath (overrides empty stub) - _PrepareNativeAssemblySources (overrides empty stub) - _AddLegacyTypeManagerResources (TypeManager.java + mono.android.jar) - _RemoveRegisterAttribute (full version with RemoveRegisterAttribute task) - _CollectLegacyTypeMapFiles (FastDev archive) - New: Xamarin.Android.TypeMap.Trimmable.targets — dispatcher + validation stub - New: Xamarin.Android.TypeMap.Trimmable.CoreCLR.targets — empty stub - New: Xamarin.Android.TypeMap.Trimmable.NativeAOT.targets — empty stub - Modified: Common.targets — stripped of all legacy typemap task invocations, added empty stubs + conditional imports - Modified: ILLink.targets — added conditions to skip legacy ILLink steps when _AndroidTypeMapImplementation == 'trimmable' Fixes: #10779
…erateLegacyAndroidManifest GenerateMainAndroidManifest unregisters NativeCodeGenState from the build engine, so GenerateTypeMappings must retrieve it first. Add DependsOnTargets to ensure correct ordering between the two AfterTargets targets.
Move _PrepareLinking, _FixRootAssembly, and _TouchAndroidLinkFlag from Microsoft.Android.Sdk.ILLink.targets into Xamarin.Android.TypeMap.Legacy.targets. This eliminates scattered _AndroidTypeMapImplementation != 'trimmable' conditions in ILLink.targets — the conditions are now implicit since Legacy.targets is only imported when _AndroidTypeMapImplementation is not 'trimmable'.
Address PR feedback: - Rename Xamarin.Android.TypeMap.*.targets -> Microsoft.Android.Sdk.TypeMap.*.targets - Move files to Microsoft.Android.Sdk/targets/ (new file convention) - Remove @(None) entries (handled by _CopyNetSdkTargets wildcard) - Delete Microsoft.Android.Sdk.ILLink.targets (moved _LinkAssemblies to Common.targets) - Add _ValidateAndroidTypeMapImplementation target - Add _GenerateJavaStubs stub in Trimmable.targets - Fix UsingTask references to use $(_XamarinAndroidBuildTasksAssembly) - Fix ILLink DLL paths for targets/ directory context
…e stub - Restore missing TypeMappingStep ILLink custom step for managed typemap implementation (used by NativeAOT) - Add proper DependsOnTargets, Inputs/Outputs, and stamp file to the trimmable _GenerateJavaStubs stub for incremental build support - Add TODO comment about manifest extraction (#10807): both GenerateMainAndroidManifest and GenerateAdditionalProviderSources depend on NativeCodeGenState and need to be decoupled before they can be shared across typemap paths
Implement core building blocks for issue #10807: Phase 1: ACW map generation - AcwMapWriter: generates acw-map.txt from scanner results with XA4214/XA4215 conflict detection - TrimmableTypeMapGenerator: per-assembly pipeline orchestrator (scan → TypeMap → JCW → ACW map → component data) Phase 2: IManifestTypeInfo abstraction - IManifestTypeInfo interface: decouples manifest generation from Cecil - ManifestTypeInfo: default mutable implementation - ManifestComponentKind enum: Activity, Service, BroadcastReceiver, ContentProvider, Application, Instrumentation - ComponentAttributeInfo: raw property dictionaries for attributes Phase 3: Component attribute extraction - ComponentAttributeExtractor: SRM-based extraction of component attributes ([Activity], [Service], etc.) and sub-attributes ([IntentFilter], [MetaData], [Layout], [Property], [GrantUriPermission]) - Integrated into JavaPeerScanner (ComponentData on JavaPeerInfo) - ScannerManifestTypeInfoAdapter: converts scanner output to IManifestTypeInfo objects - ComponentDataSerializer: text-based serialization for inter-target communication of component data Tests: 346 tests (28 new) covering all new code: - AcwMapWriterTests (14 tests) - ManifestTypeInfoTests (12 tests) - ComponentAttributeExtractorTests (19 tests) - ScannerManifestTypeInfoAdapterTests (16 tests) - EndToEndTests (10 tests) - ComponentDataSerializerTests (19 tests) - TrimmableTypeMapGeneratorTests (9 tests)
…apGenerator Remove 1,280 lines of unnecessary code: - ComponentDataSerializer (349 lines) + tests (512 lines): custom text serializer with no consumer — ScannerManifestTypeInfoAdapter already converts JavaPeerInfo → IManifestTypeInfo directly - TrimmableTypeMapGenerator (118 lines) + tests (212 lines): pipeline orchestrator with no caller — individual pieces are independently tested Also trim verbose doc comments and simplify null-coalescing in ScannerManifestTypeInfoAdapter. All 318 tests pass.
01136d4 to
fedc90f
Compare
1b05635 to
8287fe9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #10807
Stacked on #10808
Summary
Decomposes the monolithic
_GenerateJavaStubsmega-target to extract shared tasks (ACW map generation, manifest generation, provider sources) that both the legacy LLVM IR path and the new trimmable typemap path need.What's included
IManifestTypeInfoabstraction (~40 lines)Interface decoupling manifest generation from Cecil. Implemented by both
CecilManifestTypeInfoAdapter(legacy) andScannerManifestTypeInfoAdapter(trimmable). Compiled into both projects via<Compile Include>.ManifestXmlGenerator(~264 lines)Generates AndroidManifest.xml XElements from
ComponentAttributeInfodata without Cecil. Maps ~90 C# attribute properties toandroid:xxxXML attributes. Handles IntentFilter constructor args, data elements, layout, meta-data, property, and grant-uri-permission sub-elements.ManifestDocument.Merge()dual-pathModified to accept
IReadOnlyList<IManifestTypeInfo>instead ofList<TypeDefinition>. Cecil path casts toCecilManifestTypeInfoAdapterfor the existing pipeline. Non-Cecil path usesManifestXmlGenerator.GenerateTrimmableJavaStubsMSBuild task (~141 lines)Replaces
GenerateJavaStubs + GenerateMainAndroidManifestfor the trimmable path. UsesJavaPeerScannerfor scanning,AcwMapWriterfor ACW map,ScannerManifestTypeInfoAdapterfor conversion.Trimmable.targetswiringFully wired with
GenerateTrimmableJavaStubs+GenerateAdditionalProviderSources. Stamp-based incrementality,_ManifestOutput,_MergedManifestDocuments,FileWrites.Supporting types
AcwMapWriter— generatesacw-map.txtfrom scanner resultsUnconditionalMarker— marks types from customview-map and manifest as unconditionalManifestTypeReader— readsandroid:namefrom merged manifestCecilManifestTypeInfoAdapter— wraps CecilTypeDefinition→IManifestTypeInfoTests (349 total, ~1k lines new)
Diff vs base branch
~1,059 lines added (production + tests for this issue only)