From 183daf4cef4bf77c3c4b4b1f9cd0e83191282616 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Tue, 18 Aug 2026 15:32:39 +0200 Subject: [PATCH 01/11] Share the contents imported from a referenced assembly between projects Two projects that reference the same assembly import it twice: each builds its own Entity graph for every type it touches, its own EntityRefs, and for an F# assembly its own unpickling of the whole signature blob. In a solution whose projects reference the same binaries this is the largest duplicated cost - 548 MB across ten projects of ReSharper.FSharp, which reference ~500 dlls each. An assembly's imported form can be shared only if everything it can reach is identical too, because its contents point at the CcuThunks of its own closure, and because the per-CCU caches it carries - such as CSharpStyleExtensionMembersCache - hold TyconRefs into those. So each assembly gets a key covering the file, that whole closure, and the configuration that affects how types are imported; assemblies whose closures differ get separate entries. Enabled per checker with FSharpChecker.Create(shareImportedAssemblies = true), off by default, reaching the import path as tcConfig.shareImportedAssemblies. Never shared: a project's own output, a type provider assembly, a multi-module assembly, one whose simple name is claimed twice in a batch, and anything whose closure the batch cannot account for. A cached ccu may only depend on what its key pins, so it closes over the framework layer - pinned by stamp and already shared by FrameworkImportsCache - and the ccus of its own closure, never the TcImports of whichever project imported it first. That took removing four captures: MemberSignatureEquality now reaches TcGlobals through the framework layer, and TypeForwarders, ImportProvidedType and the amap thunk use an import context resolving assembly references against the entry's own closure. Entries are held weakly. A cached ccu needs no help staying alive - the projects using it hold it through their TcImports, and it holds the rest of its own closure - so an entry lives exactly as long as it is wanted, and FSharpChecker.ClearCaches drops the lot. Assemblies are identified by full path and last write time, as ILModuleReaderCacheKey does: ILAssemblyRef.QualifiedName cannot separate the builds of one package for different target frameworks, and a solution mixing frameworks references several at once. Retained memory, every project of the solution checked and held, each solution built as a real options graph: ReSharper.FSharp 10 proj 1545.3 -> 997.5 MB -547.8 (-35.5%) Oxpecker 16 proj 226.9 -> 190.9 MB -36.0 (-15.9%) FsToolkit 8 proj 218.3 -> 206.8 MB -11.5 (-5.3%) IcedTasks 7 proj 194.4 -> 184.3 MB -10.2 (-5.2%) Fantomas 8 proj 664.6 -> 657.0 MB -7.5 (-1.1%) FCS solution 14 proj 2452.6 -> 2436.7 MB -16.0 (-0.7%) consoleapp 1 proj 30.6 -> 30.7 MB +0.1 - nothing to share What separates the ends of that range is how much non-framework dll a solution references: sibling projects arrive as FSharpReference and are out of scope here, and BCL assemblies are already shared by FrameworkImportsCache. Every diagnostic of every severity, sorted, is identical with sharing off and on across those solutions. Co-Authored-By: Claude Opus 5 --- src/Compiler/Driver/CompilerConfig.fs | 6 + src/Compiler/Driver/CompilerConfig.fsi | 6 + src/Compiler/Driver/CompilerImports.fs | 464 +++++++++++++++++-- src/Compiler/Driver/CompilerImports.fsi | 9 + src/Compiler/Service/BackgroundCompiler.fs | 2 + src/Compiler/Service/BackgroundCompiler.fsi | 1 + src/Compiler/Service/IncrementalBuild.fs | 2 + src/Compiler/Service/IncrementalBuild.fsi | 1 + src/Compiler/Service/TransparentCompiler.fs | 3 + src/Compiler/Service/TransparentCompiler.fsi | 1 + src/Compiler/Service/service.fs | 10 + src/Compiler/Service/service.fsi | 2 + 12 files changed, 460 insertions(+), 47 deletions(-) diff --git a/src/Compiler/Driver/CompilerConfig.fs b/src/Compiler/Driver/CompilerConfig.fs index c1da8d3c1fd..3e1eac8134a 100644 --- a/src/Compiler/Driver/CompilerConfig.fs +++ b/src/Compiler/Driver/CompilerConfig.fs @@ -645,6 +645,10 @@ type TcConfigBuilder = mutable parallelReferenceResolution: ParallelReferenceResolution + /// Whether the Entity graph imported from a referenced assembly may be shared with other projects + /// that resolve that assembly, and everything it can reach, to the same files + mutable shareImportedAssemblies: bool + mutable captureIdentifiersWhenParsing: bool mutable typeCheckingConfig: TypeCheckingConfig @@ -844,6 +848,7 @@ type TcConfigBuilder = xmlDocInfoLoader = None exiter = QuitProcessExiter parallelReferenceResolution = ParallelReferenceResolution.On + shareImportedAssemblies = false captureIdentifiersWhenParsing = false typeCheckingConfig = { @@ -1397,6 +1402,7 @@ type TcConfig private (data: TcConfigBuilder, validate: bool) = member _.xmlDocInfoLoader = data.xmlDocInfoLoader member _.exiter = data.exiter member _.parallelReferenceResolution = data.parallelReferenceResolution + member _.shareImportedAssemblies = data.shareImportedAssemblies member _.captureIdentifiersWhenParsing = data.captureIdentifiersWhenParsing member _.typeCheckingConfig = data.typeCheckingConfig member _.dumpSignatureData = data.dumpSignatureData diff --git a/src/Compiler/Driver/CompilerConfig.fsi b/src/Compiler/Driver/CompilerConfig.fsi index 89731f6decc..60528944861 100644 --- a/src/Compiler/Driver/CompilerConfig.fsi +++ b/src/Compiler/Driver/CompilerConfig.fsi @@ -515,6 +515,10 @@ type TcConfigBuilder = mutable parallelReferenceResolution: ParallelReferenceResolution + /// Whether the Entity graph imported from a referenced assembly may be shared with other projects + /// that resolve that assembly, and everything it can reach, to the same files + mutable shareImportedAssemblies: bool + mutable captureIdentifiersWhenParsing: bool mutable typeCheckingConfig: TypeCheckingConfig @@ -888,6 +892,8 @@ type TcConfig = member parallelReferenceResolution: ParallelReferenceResolution + member shareImportedAssemblies: bool + member captureIdentifiersWhenParsing: bool member typeCheckingConfig: TypeCheckingConfig diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index e9e5c2d4206..949d23205bc 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -15,6 +15,7 @@ open System.Reflection open Internal.Utilities open Internal.Utilities.Collections open Internal.Utilities.FSharpEnvironment +open Internal.Utilities.Hashing open Internal.Utilities.Library open Internal.Utilities.Library.Extras @@ -430,6 +431,133 @@ type AssemblyResolution = this.ilAssemblyRef <- Some assemblyRef assemblyRef +/// Shares the imported form of an assembly between projects. +/// +/// Two projects that resolve an assembly, and everything it can reach, to the same files can share its +/// Entity graph. Its edges point at the CcuThunks of its own closure, so the key covers all of that. +module internal SharedImportedCcus = + + /// Weak: an entry is held by the projects using it, and holds the rest of its own closure + let private cache = ConcurrentDictionary>() + + /// Assembly names are compared case insensitively, as TcImports compares them + let private nameComparer = StringComparer.OrdinalIgnoreCase + + /// What a shared ccu may close over besides its own closure. Functions because TcImports comes later. + type FrameworkLayer = + { + Globals: unit -> TcGlobals + Resolve: CompilationThreadToken * range * ILAssemblyRef -> CcuResolutionResult + XmlDoc: string -> XmlDocumentationInfo option + } + + /// What a shared ccu resolves through, in place of the TcImports of whichever project imported it + /// first: the ccus of its own closure, then the framework layer - both pinned by its key. Holding + /// those closure ccus strongly is what lets the cache be weak. + type SharedImportContext(layer: FrameworkLayer) = + + let refs = ConcurrentDictionary(nameComparer) + + let pending = ResizeArray() + + let loader = + { new AssemblyLoader with + member _.FindCcuFromAssemblyRef(ctok, m, ilAssemblyRef: ILAssemblyRef) = + match refs.TryGetValue ilAssemblyRef.Name with + | true, ccu -> ResolvedCcu ccu + | _ -> layer.Resolve(ctok, m, ilAssemblyRef) + + member _.TryFindXmlDocumentationInfo assemblyName = layer.XmlDoc assemblyName + +#if !NO_TYPEPROVIDERS + // Type provider assemblies are never shared, so reaching these means that exclusion broke + member _.GetProvidedAssemblyInfo(_ctok, m, _assembly) = + error (InternalError("a shared ccu cannot import provided types", m)) + + member _.RecordGeneratedTypeRoot _root = + error (InternalError("a shared ccu cannot record a generated type root", range0)) +#endif + } + + let importMap = lazy ImportMap(layer.Globals(), loader) + + member _.AddClosureRef(name: string, ccu: CcuThunk) = refs[name] <- ccu + + member _.GetImportMap() = importMap.Force() + + /// Publishing before the closure is known would let another project take an entry that resolves + /// nothing + member _.HoldForPublication(key: string, ccu: CcuThunk) = pending.Add(key, ccu) + + member _.Pending = pending + + let private closureOf (assemblies: Dictionary) root = + let inside = HashSet(nameComparer) + let outside = HashSet(nameComparer) + + let rec walk n = + if inside.Add n then + for r in snd assemblies[n] do + if assemblies.ContainsKey r then walk r else outside.Add r |> ignore + + walk root + inside, outside + + /// A key per shareable assembly, covering its whole closure, so two projects meet at an entry only when + /// everything behind it resolved to the same file too. `assemblies` maps each simple name in the batch + /// to the file it resolved to and the names it can reach; names leaving it are framework assemblies, + /// which the configuration key pins. + /// + /// An assembly whose closure is not shareable gets no key: its entities, and the per-CCU caches they + /// carry, can point into one project's copy of an unshared assembly. + let computeKeys configKey (assemblies: Dictionary) isShareable = + let keys = Dictionary(nameComparer) + + for KeyValue(name, _) in assemblies do + let inside, outside = closureOf assemblies name + + if Seq.forall isShareable inside then + let parts = + Seq.append (inside |> Seq.map (fun n -> n + "|" + fst assemblies[n])) (outside |> Seq.map (fun r -> "unresolved:" + r)) + |> Seq.sort + + keys[name] <- + Md5StringHasher.empty + |> Md5StringHasher.addString configKey + |> Md5StringHasher.addStrings parts + + keys + + let private tryGet (key: string) = + match cache.TryGetValue key with + | true, wr -> + match wr.TryGetTarget() with + | true, ccu -> Some ccu + | _ -> + cache.TryRemove key |> ignore + None + | _ -> None + + /// The shared entry for an assembly, or a fresh one from `build` held until its closure is known. + /// `build` is not called on a hit, which is what lets the F# path skip unpickling entirely. + let getOrBuild (entry: (string * SharedImportContext) option) (build: unit -> CcuThunk) = + match entry with + | None -> build () + | Some(key, ctx) -> + match tryGet key with + | Some ccu -> ccu + | None -> + let ccu = build () + ctx.HoldForPublication(key, ccu) + ccu + + /// Last writer wins: each of two projects importing at once keeps the ccu it built, and both are + /// internally consistent + let add (key: string) (ccu: CcuThunk) = + cache[key] <- WeakReference ccu + + let clear () = cache.Clear() + type ImportedBinary = { FileName: string @@ -1223,6 +1351,10 @@ and [] TcImports let tciLock = TcImportsLock() + /// Identifies this instance in cache keys: an identity hash is neither unique nor stable across + /// collection, and aliasing two import layers would silently mix their ccus + let stamp = newStamp () + //---- Start protected by tciLock ------- let mutable resolutions = initialResolutions let mutable dllInfos: ImportedBinary list = [] @@ -1364,6 +1496,14 @@ and [] TcImports | Some importsBase -> importsBase.AllAssemblyResolutions() @ ars | None -> ars) + member _.Stamp = stamp + + /// The layer a sharing key pins by stamp, and so the only one a shared ccu may close over + member tcImports.KeyPinnedLayer = + match importsBase with + | Some b -> b + | None -> tcImports + member tcImports.TryFindDllInfo(ctok: CompilationThreadToken, m, assemblyName, lookupOnly) = CheckDisposed() @@ -2059,28 +2199,55 @@ and [] TcImports // Compact Framework binaries must use this. However it is not // clear when else it is required, e.g. for Mono. - member tcImports.PrepareToImportReferencedILAssembly(ctok, m, fileName, dllinfo: ImportedBinary) = + member tcImports.PrepareToImportReferencedILAssembly + (ctok, m, fileName, dllinfo: ImportedBinary, ?shared: string * SharedImportedCcus.SharedImportContext) + = CheckDisposed() let tcConfig = tcConfigP.Get ctok assert dllinfo.RawMetadata.TryGetILModuleDef().IsSome let ilModule = dllinfo.RawMetadata.TryGetILModuleDef().Value let ilScopeRef = dllinfo.ILScopeRef - let auxModuleLoader = tcImports.MkLoaderForMultiModuleILAssemblies ctok m let invalidateCcu = Event<_>() - let ccu = + let sharedContext = shared |> Option.map snd + + // Everything the imported ccu later resolves goes through this thunk, so it decides whether the ccu + // is bound to the importing project or to what the sharing key pins + let amap = + match sharedContext with + | Some ctx -> ctx.GetImportMap + | None -> tcImports.GetImportMap + + // Multi-module assemblies are excluded from sharing, so a shared import can never need this + let auxModuleLoader = + match sharedContext with + | Some _ -> + fun scoref -> + error (InternalError(sprintf "a shared ccu cannot load the auxiliary module %A" scoref, m)) + | None -> tcImports.MkLoaderForMultiModuleILAssemblies ctok m + + // Meaningless in a ccu handed to other projects: SymbolHelpers.fileNameOfItem combines it with a + // relative path out of the assembly's metadata + let sourceDir = + match shared with + | Some _ -> "" + | None -> tcConfig.implicitIncludeDir + + let importFresh () = ImportILAssembly( - tcImports.GetImportMap, + amap, m, auxModuleLoader, tcConfig.xmlDocInfoLoader, ilScopeRef, - tcConfig.implicitIncludeDir, + sourceDir, Some fileName, ilModule, invalidateCcu.Publish ) + let ccu = SharedImportedCcus.getOrBuild shared importFresh + let ccuinfo = { FSharpViewOfMetadata = ccu @@ -2107,8 +2274,21 @@ and [] TcImports phase2 - member tcImports.PrepareToImportReferencedFSharpAssembly(ctok, m, fileName, dllinfo: ImportedBinary) = + member tcImports.PrepareToImportReferencedFSharpAssembly + (ctok, m, fileName, dllinfo: ImportedBinary, ?shared: string * SharedImportedCcus.SharedImportContext) + = CheckDisposed() + + let sharedContext = shared |> Option.map snd + + // GetTcGlobals would reach the same object - the field is only set on the framework layer - but + // through a project's TcImports, which a cached ccu must not hold + let globalsOwner = tcImports.KeyPinnedLayer + + let amap = + match sharedContext with + | Some ctx -> ctx.GetImportMap + | None -> tcImports.GetImportMap #if !NO_TYPEPROVIDERS let tcConfig = tcConfigP.Get ctok #endif @@ -2122,54 +2302,66 @@ and [] TcImports let ccuRawDataAndInfos = ilModule.GetRawFSharpSignatureData(m, ilShortAssemName, fileName) |> List.map (fun (ccuName, (sigDataReader, sigDataReaderB)) -> - let data = - GetSignatureData(fileName, ilScopeRef, ilModule.TryGetILModuleDef(), sigDataReader, sigDataReaderB) + // One entry per ccu, because an assembly can carry several + let entry = + shared |> Option.map (fun (key, ctx) -> key + "|fsharp|" + ccuName, ctx) let optDatas = Map.ofList optDataReaders - let minfo: PickledCcuInfo = data.RawData - let mspec = minfo.mspec - - if mspec.DisplayName = "FSharp.Core" then - updateSeqTypeIsPrefix mspec - #if !NO_TYPEPROVIDERS let invalidateCcu = Event<_>() #endif - let codeDir = minfo.compileTimeWorkingDir + // None on a hit: nothing is left to relink, and the signature data is never read + let mutable dataOpt = None - // note: for some fields we fix up this information later - let ccuData: CcuData = - { - ILScopeRef = ilScopeRef - Stamp = newStamp () - FileName = Some fileName - QualifiedName = Some(ilScopeRef.QualifiedName) - SourceCodeDirectory = codeDir - IsFSharp = true - Contents = mspec + let ccu = + SharedImportedCcus.getOrBuild entry (fun () -> + let data = + GetSignatureData(fileName, ilScopeRef, ilModule.TryGetILModuleDef(), sigDataReader, sigDataReaderB) + + let minfo: PickledCcuInfo = data.RawData + let mspec = minfo.mspec + + if mspec.DisplayName = "FSharp.Core" then + updateSeqTypeIsPrefix mspec + + let codeDir = minfo.compileTimeWorkingDir + + // note: for some fields we fix up this information later + let ccuData: CcuData = + { + ILScopeRef = ilScopeRef + Stamp = newStamp () + FileName = Some fileName + QualifiedName = Some(ilScopeRef.QualifiedName) + SourceCodeDirectory = codeDir + IsFSharp = true + Contents = mspec #if !NO_TYPEPROVIDERS - InvalidateEvent = invalidateCcu.Publish - IsProviderGenerated = false - ImportProvidedType = (fun ty -> ImportProvidedType (tcImports.GetImportMap()) m ty) + InvalidateEvent = invalidateCcu.Publish + IsProviderGenerated = false + ImportProvidedType = (fun ty -> ImportProvidedType (amap ()) m ty) #endif - TryGetILModuleDef = ilModule.TryGetILModuleDef - UsesFSharp20PlusQuotations = minfo.usesQuotations - MemberSignatureEquality = (fun ty1 ty2 -> typeEquivAux EraseAll (tcImports.GetTcGlobals()) ty1 ty2) - TypeForwarders = ImportILAssemblyTypeForwarders(tcImports.GetImportMap, m, ilModule.GetRawTypeForwarders()) - CSharpStyleExtensionMembersCache = ConcurrentDictionary(1, 0) + TryGetILModuleDef = ilModule.TryGetILModuleDef + UsesFSharp20PlusQuotations = minfo.usesQuotations + MemberSignatureEquality = + (fun ty1 ty2 -> typeEquivAux EraseAll (globalsOwner.GetTcGlobals()) ty1 ty2) + TypeForwarders = + ImportILAssemblyTypeForwarders(amap, m, ilModule.GetRawTypeForwarders()) + CSharpStyleExtensionMembersCache = ConcurrentDictionary(1, 0) #if !NO_TYPEPROVIDERS - XmlDocumentationInfo = - match tcConfig.xmlDocInfoLoader with - | Some xmlDocInfoLoader -> xmlDocInfoLoader.TryLoad(fileName) - | _ -> None + XmlDocumentationInfo = + match tcConfig.xmlDocInfoLoader with + | Some xmlDocInfoLoader -> xmlDocInfoLoader.TryLoad(fileName) + | _ -> None #else - XmlDocumentationInfo = None + XmlDocumentationInfo = None #endif - } + } - let ccu = CcuThunk.Create(ccuName, ccuData) + dataOpt <- Some data + CcuThunk.Create(ccuName, ccuData)) let optdata = InterruptibleLazy(fun _ -> @@ -2228,7 +2420,7 @@ and [] TcImports #else () #endif - data, ccuinfo, phase2) + dataOpt, ccuinfo, phase2) // Register all before relinking to cope with mutually-referential ccus ccuRawDataAndInfos |> List.iter (p23 >> tcImports.RegisterCcu) @@ -2236,7 +2428,12 @@ and [] TcImports let phase2 () = // Relink ccuRawDataAndInfos - |> List.iter (fun (data, _, _) -> + |> List.iter (fun (dataOpt, _, _) -> + // A shared ccu was relinked by whichever project unpickled it, against the same closure. + match dataOpt with + | None -> () + | Some data -> + let fixupThunk () = data.OptionalFixup(fun nm -> availableToOptionalCcu (tcImports.FindCcu(ctok, m, nm, lookupOnly = false))) |> ignore @@ -2258,6 +2455,15 @@ and [] TcImports // NOTE: When used in the Language Service this can cause the transitive checking of projects. Hence it must be cancellable. member tcImports.RegisterAndImportReferencedAssemblies(ctok, nms: AssemblyResolution list) = + let frameworkLayer: SharedImportedCcus.FrameworkLayer = + let layer = tcImports.KeyPinnedLayer + + { + Globals = layer.GetTcGlobals + Resolve = fun (ctok, m, aref) -> layer.FindCcuFromAssemblyRef(ctok, m, aref) + XmlDoc = layer.TryFindXmlDocumentationInfo + } + let tryGetAssemblyData (r: AssemblyResolution) = async { CheckDisposed() @@ -2290,12 +2496,148 @@ and [] TcImports return None } - let registerDll (r: AssemblyResolution, assemblyData: IRawFSharpAssemblyData) = + /// Every assembly name a piece of metadata can reach: its own, the ones it references, and the + /// targets of its type forwarders, which is all a facade's closure consists of. Also reports a + /// multi-module assembly, whose auxiliary modules need a loader bound to the importing project. + let reachableAssemblyNames (self: string) (data: IRawFSharpAssemblyData) = + let names = ResizeArray() + names.Add self + + for aref in data.ILAssemblyRefs do + names.Add aref.Name + + let mutable multiModule = false + + match data.TryGetILModuleDef() |> Option.bind (fun ilModule -> ilModule.Manifest) with + | Some manifest -> + for e in manifest.ExportedTypes.AsList() do + match e.ScopeRef with + | ILScopeRef.Assembly aref -> names.Add aref.Name + | ILScopeRef.Module _ -> multiModule <- true + | _ -> () + | None -> () + + List.distinct (List.ofSeq names), multiModule + + let isTypeProviderAssembly (data: IRawFSharpAssemblyData) = + match data.TryGetILModuleDef() with + | Some ilModule -> + ilModule.ManifestOfAssembly.CustomAttrs.AsList() + |> List.exists (fun a -> a.Method.DeclaringType.BasicQualifiedName.Contains "TypeProviderAssembly") + | None -> false + + /// The file an assembly resolved to, as ILModuleReaderCacheKey identifies one. ILAssemblyRef cannot + /// separate one package's builds for different target frameworks, which differ only in content. + let fileIdentity (r: AssemblyResolution) = + let writeStamp = + try + string (FileSystem.GetLastWriteTimeShim r.resolvedPath).Ticks + with _ -> + "nostamp" + + r.resolvedPath + "|" + writeStamp + + let sharedKeys (all: (AssemblyResolution * IRawFSharpAssemblyData) list) = + let tcConfig = tcConfigP.Get ctok + let ic = StringComparer.OrdinalIgnoreCase + let short (p: string) = Path.GetFileNameWithoutExtension p + + // Anything that changes how types are imported belongs here. The framework layer goes in by + // stamp, since FrameworkImportsCache already shares it per configuration. + let configKey = + sprintf + "%d|%b|%O|%b" + (match importsBase with + | Some b -> b.Stamp + | None -> 0L) + tcConfig.checkNullness + tcConfig.langVersion.SpecifiedVersion + tcConfig.xmlDocInfoLoader.IsSome + + // Two resolutions claiming one name would key whichever was seen last, so neither is shared + let ambiguous = + all + |> List.countBy (fun (r, _) -> short r.resolvedPath) + |> List.choose (fun (name, n) -> if n > 1 then Some name else None) + |> fun names -> HashSet(names, ic) + + let assemblies = Dictionary(ic) + let shareable = HashSet(ic) + + for r, data in all do + let name = short r.resolvedPath + let refs, isMultiModule = reachableAssemblyNames name data + assemblies[name] <- fileIdentity r, refs + + // A project's own output changes with every build, and phase2 adds provided namespaces into + // a type provider assembly's contents + if r.ProjectReference.IsNone + && not isMultiModule + && not (ambiguous.Contains name) + && not (isTypeProviderAssembly data) then + shareable.Add name |> ignore + + // A reference must resolve to what the key pins - this batch or the framework layer - or to + // nothing anywhere, which every sharer agrees about. One only an earlier batch of this project + // resolves would be keyed as unresolved while this project resolves it. + let pinnedByKey = Dictionary(ic) + + let isPinnedByKey ref = + match pinnedByKey.TryGetValue ref with + | true, v -> v + | _ -> + let resolvesIn (t: TcImports) = + match t.FindCcu(ctok, range0, ref, lookupOnly = true) with + | ResolvedCcu _ -> true + | UnresolvedCcu _ -> false + + let v = + assemblies.ContainsKey ref + || (match importsBase with + | Some b -> resolvesIn b + | None -> false) + || not (resolvesIn tcImports) + + pinnedByKey[ref] <- v + v + + for name in List.ofSeq shareable do + if not (snd assemblies[name] |> List.forall isPinnedByKey) then + shareable.Remove name |> ignore + + // The closure travels with the key: it is what the entry's import context has to resolve + let shared = Dictionary(ic) + + for KeyValue(name, key) in SharedImportedCcus.computeKeys configKey assemblies shareable.Contains do + shared[name] <- key, snd assemblies[name] + + shared + + let contexts = ResizeArray() + + let registerDll + (keys: Dictionary option) + (r: AssemblyResolution, assemblyData: IRawFSharpAssemblyData) + = let m = r.originalReference.Range let fileName = r.resolvedPath let ilShortAssemName = assemblyData.ShortAssemblyName let ilScopeRef = assemblyData.ILScopeRef + // A project's own output is never shared, and can share a simple name with a package, so it is + // checked here rather than left to the key table + let shared = + match keys with + | Some keys when r.ProjectReference.IsNone -> + match keys.TryGetValue(Path.GetFileNameWithoutExtension fileName) with + | true, (k, refs) -> + let ctx = SharedImportedCcus.SharedImportContext frameworkLayer + + contexts.Add(ctx, refs) + Some(k, ctx) + | _ -> None + | _ -> None + if tcImports.IsAlreadyRegistered ilShortAssemName then let phase2 () = @@ -2322,14 +2664,14 @@ and [] TcImports if assemblyData.HasAnyFSharpSignatureDataAttribute then if not assemblyData.HasMatchingFSharpSignatureDataAttribute then errorR (Error(FSComp.SR.buildDifferentVersionMustRecompile fileName, m)) - tcImports.PrepareToImportReferencedILAssembly(ctok, m, fileName, dllinfo) + tcImports.PrepareToImportReferencedILAssembly(ctok, m, fileName, dllinfo, ?shared = shared) else try - tcImports.PrepareToImportReferencedFSharpAssembly(ctok, m, fileName, dllinfo) + tcImports.PrepareToImportReferencedFSharpAssembly(ctok, m, fileName, dllinfo, ?shared = shared) with e -> error (Error(FSComp.SR.buildErrorOpeningBinaryFile (fileName, e.Message), m)) else - tcImports.PrepareToImportReferencedILAssembly(ctok, m, fileName, dllinfo) + tcImports.PrepareToImportReferencedILAssembly(ctok, m, fileName, dllinfo, ?shared = shared) async { return phase2 () } @@ -2346,12 +2688,40 @@ and [] TcImports let! assemblyData = nms |> List.map tryGetAssemblyData |> runMethod // Preserve determinicstic order of references, because types from later assemblies may shadow earlier ones. - let phase2s = assemblyData |> Seq.choose id |> Seq.map registerDll |> List.ofSeq + let resolved = assemblyData |> Seq.choose id |> List.ofSeq + + let keys = + // A framework base exists only for project layers, which register their whole non-framework + // set at once; BuildFrameworkTcImports registers in three batches, and an entry from an + // early one could not resolve what the later ones bring. + // + // Only with reduceMemoryUsage: without it TcImports disposal closes a memory-mapped reader, + // so a ccu outliving the project that imported it would read from a closed file. + if tcConfig.shareImportedAssemblies + && importsBase.IsSome + && tcConfig.reduceMemoryUsage = ReduceMemoryFlag.Yes then + Some(sharedKeys resolved) + else + None + + let phase2s = resolved |> List.map (registerDll keys) fixupOrphanCcus () let! ccuinfos = phase2s |> runMethod + // Every assembly is registered now, so each new entry can be given its closure and published; + // nothing has forced an import yet, so nothing can have resolved through a half-filled one + for ctx, refs in contexts do + if ctx.Pending.Count > 0 then + for r in refs do + match tcImports.FindCcu(ctok, range0, r, lookupOnly = true) with + | ResolvedCcu ccu -> ctx.AddClosureRef(r, ccu) + | UnresolvedCcu _ -> () + + for key, ccu in ctx.Pending do + SharedImportedCcus.add key ccu + if importsBase.IsSome then let addConstraintSources (ia: ImportedAssembly) = // Only an F# assembly can carry a trait constraint to label. diff --git a/src/Compiler/Driver/CompilerImports.fsi b/src/Compiler/Driver/CompilerImports.fsi index 9da0ef71b1d..15ae2b06ec1 100644 --- a/src/Compiler/Driver/CompilerImports.fsi +++ b/src/Compiler/Driver/CompilerImports.fsi @@ -98,6 +98,15 @@ type ResolvedExtensionReference = | ResolvedExtensionReference of string * AssemblyReference list * Tainted list #endif +/// Holds the contents imported from a referenced assembly so that projects resolving that assembly, and +/// everything it can reach, to the same files share one copy of its Entity graph. Switched on per checker +/// by tcConfig.shareImportedAssemblies. Entries are weak, so nothing is retained on a project's behalf +/// once that project is gone. +module internal SharedImportedCcus = + + /// Drops every entry, for FSharpChecker.ClearCaches + val clear: unit -> unit + /// Represents a resolved imported binary [] type ImportedBinary = diff --git a/src/Compiler/Service/BackgroundCompiler.fs b/src/Compiler/Service/BackgroundCompiler.fs index 9e0e32bd7e7..50f9881260f 100644 --- a/src/Compiler/Service/BackgroundCompiler.fs +++ b/src/Compiler/Service/BackgroundCompiler.fs @@ -245,6 +245,7 @@ type internal BackgroundCompiler enableBackgroundItemKeyStoreAndSemanticClassification, enablePartialTypeChecking, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource: (string -> Async) option, useChangeNotifications @@ -382,6 +383,7 @@ type internal BackgroundCompiler enablePartialTypeChecking, dependencyProvider, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource, useChangeNotifications diff --git a/src/Compiler/Service/BackgroundCompiler.fsi b/src/Compiler/Service/BackgroundCompiler.fsi index 30c008c2ed5..fe098cd22be 100644 --- a/src/Compiler/Service/BackgroundCompiler.fsi +++ b/src/Compiler/Service/BackgroundCompiler.fsi @@ -232,6 +232,7 @@ type internal BackgroundCompiler = enableBackgroundItemKeyStoreAndSemanticClassification: bool * enablePartialTypeChecking: bool * parallelReferenceResolution: ParallelReferenceResolution * + shareImportedAssemblies: bool * captureIdentifiersWhenParsing: bool * getSource: (string -> Async) option * useChangeNotifications: bool -> diff --git a/src/Compiler/Service/IncrementalBuild.fs b/src/Compiler/Service/IncrementalBuild.fs index 48a42e66ff9..d3c79be7c52 100644 --- a/src/Compiler/Service/IncrementalBuild.fs +++ b/src/Compiler/Service/IncrementalBuild.fs @@ -1432,6 +1432,7 @@ type IncrementalBuilder(initialState: IncrementalBuilderInitialState, state: Inc enablePartialTypeChecking, dependencyProvider, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource, useChangeNotifications @@ -1518,6 +1519,7 @@ type IncrementalBuilder(initialState: IncrementalBuilderInitialState, state: Inc |> Some tcConfigB.parallelReferenceResolution <- parallelReferenceResolution + tcConfigB.shareImportedAssemblies <- shareImportedAssemblies tcConfigB.captureIdentifiersWhenParsing <- captureIdentifiersWhenParsing tcConfigB, sourceFilesNew diff --git a/src/Compiler/Service/IncrementalBuild.fsi b/src/Compiler/Service/IncrementalBuild.fsi index cced43ea44b..fa2b46553d3 100644 --- a/src/Compiler/Service/IncrementalBuild.fsi +++ b/src/Compiler/Service/IncrementalBuild.fsi @@ -291,6 +291,7 @@ type internal IncrementalBuilder = enablePartialTypeChecking: bool * dependencyProvider: DependencyProvider option * parallelReferenceResolution: ParallelReferenceResolution * + shareImportedAssemblies: bool * captureIdentifiersWhenParsing: bool * getSource: (string -> Async) option * useChangeNotifications: bool -> diff --git a/src/Compiler/Service/TransparentCompiler.fs b/src/Compiler/Service/TransparentCompiler.fs index 4666aa930ed..37f5cfb7c17 100644 --- a/src/Compiler/Service/TransparentCompiler.fs +++ b/src/Compiler/Service/TransparentCompiler.fs @@ -419,6 +419,7 @@ type internal TransparentCompiler enableBackgroundItemKeyStoreAndSemanticClassification, enablePartialTypeChecking, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource: (string -> Async) option, useChangeNotifications, @@ -469,6 +470,7 @@ type internal TransparentCompiler enableBackgroundItemKeyStoreAndSemanticClassification, enablePartialTypeChecking, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource, useChangeNotifications @@ -931,6 +933,7 @@ type internal TransparentCompiler |> Some tcConfigB.parallelReferenceResolution <- parallelReferenceResolution + tcConfigB.shareImportedAssemblies <- shareImportedAssemblies tcConfigB.captureIdentifiersWhenParsing <- captureIdentifiersWhenParsing return tcConfigB, sourceFilesNew, loadClosureOpt diff --git a/src/Compiler/Service/TransparentCompiler.fsi b/src/Compiler/Service/TransparentCompiler.fsi index 7e947fe81e9..2d1c3506f0b 100644 --- a/src/Compiler/Service/TransparentCompiler.fsi +++ b/src/Compiler/Service/TransparentCompiler.fsi @@ -190,6 +190,7 @@ type internal TransparentCompiler = enableBackgroundItemKeyStoreAndSemanticClassification: bool * enablePartialTypeChecking: bool * parallelReferenceResolution: ParallelReferenceResolution * + shareImportedAssemblies: bool * captureIdentifiersWhenParsing: bool * getSource: (string -> Async) option * useChangeNotifications: bool * diff --git a/src/Compiler/Service/service.fs b/src/Compiler/Service/service.fs index 1006def6da1..e2d9f6450bd 100644 --- a/src/Compiler/Service/service.fs +++ b/src/Compiler/Service/service.fs @@ -97,6 +97,7 @@ type FSharpChecker enableBackgroundItemKeyStoreAndSemanticClassification, enablePartialTypeChecking, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource, useChangeNotifications, @@ -117,6 +118,7 @@ type FSharpChecker enableBackgroundItemKeyStoreAndSemanticClassification, enablePartialTypeChecking, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource, useChangeNotifications, @@ -135,6 +137,7 @@ type FSharpChecker enableBackgroundItemKeyStoreAndSemanticClassification, enablePartialTypeChecking, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, getSource, useChangeNotifications @@ -181,6 +184,7 @@ type FSharpChecker ?enableBackgroundItemKeyStoreAndSemanticClassification, ?enablePartialTypeChecking, ?parallelReferenceResolution: bool, + ?shareImportedAssemblies: bool, ?captureIdentifiersWhenParsing: bool, ?documentSource: DocumentSource, ?useTransparentCompiler: bool, @@ -215,6 +219,8 @@ type FSharpChecker if keepAssemblyContents && enablePartialTypeChecking then invalidArg "enablePartialTypeChecking" "'keepAssemblyContents' and 'enablePartialTypeChecking' cannot be both enabled." + let shareImportedAssemblies = defaultArg shareImportedAssemblies false + let parallelReferenceResolution = inferParallelReferenceResolution parallelReferenceResolution FSharpChecker( @@ -228,6 +234,7 @@ type FSharpChecker enableBackgroundItemKeyStoreAndSemanticClassification, enablePartialTypeChecking, parallelReferenceResolution, + shareImportedAssemblies, captureIdentifiersWhenParsing, (match documentSource with | Some(DocumentSource.Custom f) -> Some f @@ -330,6 +337,9 @@ type FSharpChecker braceMatchCache.Clear(utok) backgroundCompiler.ClearCaches() ClearAllILModuleReaderCache() + // Entries are weak, so live projects keep the ones they use and the rest go on their own, but + // "clear the caches" should mean it: after this nothing is held on any project's behalf. + FSharp.Compiler.CompilerImports.SharedImportedCcus.clear () member ic.ClearLanguageServiceRootCachesAndCollectAndFinalizeAllTransients() = use _ = diff --git a/src/Compiler/Service/service.fsi b/src/Compiler/Service/service.fsi index 1584e19562b..bfa94b41605 100644 --- a/src/Compiler/Service/service.fsi +++ b/src/Compiler/Service/service.fsi @@ -32,6 +32,7 @@ type public FSharpChecker = /// Indicates whether a table of symbol keys should be kept for background compilation /// Indicates whether to perform partial type checking. Cannot be set to true if keepAssemblyContents is true. If set to true, can cause duplicate type-checks when richer information on a file is needed, but can skip background type-checking entirely on implementation files with signature files. /// Indicates whether to resolve references in parallel. + /// Default: false. Indicates whether the contents imported from a referenced assembly may be shared between projects that resolve that assembly, and everything it can reach, to the same files. Saves memory and import time in solutions where projects reference the same binaries, and costs nothing where they do not. Assemblies containing type providers, and assemblies produced by a project in the same solution, are never shared. /// When set to true we create a set of all identifiers for each parsed file which can be used to speed up finding references. /// Default: FileSystem. You can use Custom source to provide a function that will return the source for a given file path instead of reading it from the file system. Note that with this option the FSharpChecker will also not monitor the file system for file changes. It will expect to be notified of changes via the NotifyFileChanged method. /// Default: false. Indicates whether we use a new experimental background compiler. This does not yet support all features @@ -47,6 +48,7 @@ type public FSharpChecker = ?enableBackgroundItemKeyStoreAndSemanticClassification: bool * ?enablePartialTypeChecking: bool * ?parallelReferenceResolution: bool * + ?shareImportedAssemblies: bool * ?captureIdentifiersWhenParsing: bool * [] ?documentSource: DocumentSource * From ae78b62453b42a9d2f863b538758345267923519 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 19 Aug 2026 14:28:33 +0200 Subject: [PATCH 02/11] Release notes --- docs/release-notes/.FSharp.Compiler.Service/11.0.100.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/release-notes/.FSharp.Compiler.Service/11.0.100.md b/docs/release-notes/.FSharp.Compiler.Service/11.0.100.md index b30a501d979..00fae36ca5a 100644 --- a/docs/release-notes/.FSharp.Compiler.Service/11.0.100.md +++ b/docs/release-notes/.FSharp.Compiler.Service/11.0.100.md @@ -171,6 +171,7 @@ * IL: cache C# extension methods per CCU ([PR #20256](https://github.com/dotnet/fsharp/pull/20256)) * Nullness warning FS3261 on dotted method or property access (e.g. `x.Member`) now underlines the receiver expression and includes the member name and (when known) the binding name in the message. ([Issue #19658](https://github.com/dotnet/fsharp/issues/19658), [PR #19814](https://github.com/dotnet/fsharp/pull/19814)) +* Import: share assembly CCUs between projects ([PR #20296](https://github.com/dotnet/fsharp/pull/20296)) * Direct delegate construction ([PR ##19993](https://github.com/dotnet/fsharp/pull/19993)) * IL: add `ILPreNamespace`, make `ILPreTypeDef` creation lazy ([PR #20092](https://github.com/dotnet/fsharp/pull/20092)) * IL: share ILCallingConv instances ([PR #20254](https://github.com/dotnet/fsharp/pull/20254)) From eaf9708abcd4c4d0caa1242510599bd37fbf34f8 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 19 Aug 2026 15:35:32 +0200 Subject: [PATCH 03/11] Surface area --- ...piler.Service.SurfaceArea.netstandard20.bsl | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/tests/FSharp.Compiler.Service.Tests/FSharp.Compiler.Service.SurfaceArea.netstandard20.bsl b/tests/FSharp.Compiler.Service.Tests/FSharp.Compiler.Service.SurfaceArea.netstandard20.bsl index ba8d1825ece..a144171dcd5 100644 --- a/tests/FSharp.Compiler.Service.Tests/FSharp.Compiler.Service.SurfaceArea.netstandard20.bsl +++ b/tests/FSharp.Compiler.Service.Tests/FSharp.Compiler.Service.SurfaceArea.netstandard20.bsl @@ -2156,7 +2156,7 @@ FSharp.Compiler.CodeAnalysis.FSharpCheckProjectResults: System.String[] Dependen FSharp.Compiler.CodeAnalysis.FSharpCheckProjectResults: System.String[] get_DependencyFiles() FSharp.Compiler.CodeAnalysis.FSharpChecker: Boolean UsesTransparentCompiler FSharp.Compiler.CodeAnalysis.FSharpChecker: Boolean get_UsesTransparentCompiler() -FSharp.Compiler.CodeAnalysis.FSharpChecker: FSharp.Compiler.CodeAnalysis.FSharpChecker Create(Microsoft.FSharp.Core.FSharpOption`1[System.Int32], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[FSharp.Compiler.CodeAnalysis.LegacyReferenceResolver], Microsoft.FSharp.Core.FSharpOption`1[Microsoft.FSharp.Core.FSharpFunc`2[System.Tuple`2[System.String,System.DateTime],Microsoft.FSharp.Core.FSharpOption`1[System.Tuple`3[System.Object,System.IntPtr,System.Int32]]]], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[FSharp.Compiler.CodeAnalysis.DocumentSource], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[FSharp.Compiler.CodeAnalysis.TransparentCompiler.CacheSizes]) +FSharp.Compiler.CodeAnalysis.FSharpChecker: FSharp.Compiler.CodeAnalysis.FSharpChecker Create(Microsoft.FSharp.Core.FSharpOption`1[System.Int32], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[FSharp.Compiler.CodeAnalysis.LegacyReferenceResolver], Microsoft.FSharp.Core.FSharpOption`1[Microsoft.FSharp.Core.FSharpFunc`2[System.Tuple`2[System.String,System.DateTime],Microsoft.FSharp.Core.FSharpOption`1[System.Tuple`3[System.Object,System.IntPtr,System.Int32]]]], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[FSharp.Compiler.CodeAnalysis.DocumentSource], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[FSharp.Compiler.CodeAnalysis.TransparentCompiler.CacheSizes]) FSharp.Compiler.CodeAnalysis.FSharpChecker: FSharp.Compiler.CodeAnalysis.FSharpChecker Instance FSharp.Compiler.CodeAnalysis.FSharpChecker: FSharp.Compiler.CodeAnalysis.FSharpChecker get_Instance() FSharp.Compiler.CodeAnalysis.FSharpChecker: FSharp.Compiler.CodeAnalysis.FSharpProjectOptions GetProjectOptionsFromCommandLineArgs(System.String, System.String[], Microsoft.FSharp.Core.FSharpOption`1[System.DateTime], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean], Microsoft.FSharp.Core.FSharpOption`1[System.Boolean]) @@ -5667,7 +5667,6 @@ FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean EventIsStandard FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean HasGetterMethod FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean HasSetterMethod FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean HasSignatureFile -FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsPropertyAccessor FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsActivePattern FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsBaseValue FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsCompilerGenerated @@ -5690,6 +5689,7 @@ FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsModuleValueOrMe FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsMutable FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsOverrideOrExplicitInterfaceImplementation FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsProperty +FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsPropertyAccessor FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsPropertyGetterMethod FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsPropertySetterMethod FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean IsRefCell @@ -5703,7 +5703,6 @@ FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_EventIsStanda FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_HasGetterMethod() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_HasSetterMethod() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_HasSignatureFile() -FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsPropertyAccessor() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsActivePattern() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsBaseValue() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsCompilerGenerated() @@ -5726,6 +5725,7 @@ FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsModuleValue FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsMutable() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsOverrideOrExplicitInterfaceImplementation() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsProperty() +FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsPropertyAccessor() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsPropertyGetterMethod() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsPropertySetterMethod() FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue: Boolean get_IsRefCell() @@ -11332,8 +11332,8 @@ FSharp.Compiler.Text.TextTag+Tags: Int32 TypeParameter FSharp.Compiler.Text.TextTag+Tags: Int32 Union FSharp.Compiler.Text.TextTag+Tags: Int32 UnionCase FSharp.Compiler.Text.TextTag+Tags: Int32 UnknownEntity -FSharp.Compiler.Text.TextTag+Tags: Int32 UnresolvedName FSharp.Compiler.Text.TextTag+Tags: Int32 UnknownType +FSharp.Compiler.Text.TextTag+Tags: Int32 UnresolvedName FSharp.Compiler.Text.TextTag: Boolean Equals(FSharp.Compiler.Text.TextTag) FSharp.Compiler.Text.TextTag: Boolean Equals(FSharp.Compiler.Text.TextTag, System.Collections.IEqualityComparer) FSharp.Compiler.Text.TextTag: Boolean Equals(System.Object) @@ -11371,8 +11371,8 @@ FSharp.Compiler.Text.TextTag: Boolean IsTypeParameter FSharp.Compiler.Text.TextTag: Boolean IsUnion FSharp.Compiler.Text.TextTag: Boolean IsUnionCase FSharp.Compiler.Text.TextTag: Boolean IsUnknownEntity -FSharp.Compiler.Text.TextTag: Boolean IsUnresolvedName FSharp.Compiler.Text.TextTag: Boolean IsUnknownType +FSharp.Compiler.Text.TextTag: Boolean IsUnresolvedName FSharp.Compiler.Text.TextTag: Boolean get_IsActivePatternCase() FSharp.Compiler.Text.TextTag: Boolean get_IsActivePatternResult() FSharp.Compiler.Text.TextTag: Boolean get_IsAlias() @@ -11406,8 +11406,8 @@ FSharp.Compiler.Text.TextTag: Boolean get_IsTypeParameter() FSharp.Compiler.Text.TextTag: Boolean get_IsUnion() FSharp.Compiler.Text.TextTag: Boolean get_IsUnionCase() FSharp.Compiler.Text.TextTag: Boolean get_IsUnknownEntity() -FSharp.Compiler.Text.TextTag: Boolean get_IsUnresolvedName() FSharp.Compiler.Text.TextTag: Boolean get_IsUnknownType() +FSharp.Compiler.Text.TextTag: Boolean get_IsUnresolvedName() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag ActivePatternCase FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag ActivePatternResult FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag Alias @@ -11441,8 +11441,8 @@ FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag TypeParameter FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag Union FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag UnionCase FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag UnknownEntity -FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag UnresolvedName FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag UnknownType +FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag UnresolvedName FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_ActivePatternCase() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_ActivePatternResult() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_Alias() @@ -11476,8 +11476,8 @@ FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_TypeParameter() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_Union() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_UnionCase() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_UnknownEntity() -FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_UnresolvedName() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_UnknownType() +FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag get_UnresolvedName() FSharp.Compiler.Text.TextTag: FSharp.Compiler.Text.TextTag+Tags FSharp.Compiler.Text.TextTag: Int32 GetHashCode() FSharp.Compiler.Text.TextTag: Int32 GetHashCode(System.Collections.IEqualityComparer) @@ -12727,4 +12727,4 @@ Internal.Utilities.Library.InterruptibleLazy`1[T]: Internal.Utilities.Library.In Internal.Utilities.Library.InterruptibleLazy`1[T]: T Force() Internal.Utilities.Library.InterruptibleLazy`1[T]: T Value Internal.Utilities.Library.InterruptibleLazy`1[T]: T get_Value() -Internal.Utilities.Library.InterruptibleLazy`1[T]: Void .ctor(Microsoft.FSharp.Core.FSharpFunc`2[Microsoft.FSharp.Core.Unit,T]) +Internal.Utilities.Library.InterruptibleLazy`1[T]: Void .ctor(Microsoft.FSharp.Core.FSharpFunc`2[Microsoft.FSharp.Core.Unit,T]) \ No newline at end of file From 2bf816dafc490e2612e42f047cbe320a2934e341 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 19 Aug 2026 14:40:17 +0200 Subject: [PATCH 04/11] Fantomas --- src/Compiler/Driver/CompilerImports.fs | 54 +++++++++++++------------- 1 file changed, 28 insertions(+), 26 deletions(-) diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index 949d23205bc..8ef9d7da214 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -498,7 +498,10 @@ module internal SharedImportedCcus = let rec walk n = if inside.Add n then for r in snd assemblies[n] do - if assemblies.ContainsKey r then walk r else outside.Add r |> ignore + if assemblies.ContainsKey r then + walk r + else + outside.Add r |> ignore walk root inside, outside @@ -2221,9 +2224,7 @@ and [] TcImports // Multi-module assemblies are excluded from sharing, so a shared import can never need this let auxModuleLoader = match sharedContext with - | Some _ -> - fun scoref -> - error (InternalError(sprintf "a shared ccu cannot load the auxiliary module %A" scoref, m)) + | Some _ -> fun scoref -> error (InternalError(sprintf "a shared ccu cannot load the auxiliary module %A" scoref, m)) | None -> tcImports.MkLoaderForMultiModuleILAssemblies ctok m // Meaningless in a ccu handed to other projects: SymbolHelpers.fileNameOfItem combines it with a @@ -2303,8 +2304,7 @@ and [] TcImports ilModule.GetRawFSharpSignatureData(m, ilShortAssemName, fileName) |> List.map (fun (ccuName, (sigDataReader, sigDataReaderB)) -> // One entry per ccu, because an assembly can carry several - let entry = - shared |> Option.map (fun (key, ctx) -> key + "|fsharp|" + ccuName, ctx) + let entry = shared |> Option.map (fun (key, ctx) -> key + "|fsharp|" + ccuName, ctx) let optDatas = Map.ofList optDataReaders @@ -2345,10 +2345,8 @@ and [] TcImports #endif TryGetILModuleDef = ilModule.TryGetILModuleDef UsesFSharp20PlusQuotations = minfo.usesQuotations - MemberSignatureEquality = - (fun ty1 ty2 -> typeEquivAux EraseAll (globalsOwner.GetTcGlobals()) ty1 ty2) - TypeForwarders = - ImportILAssemblyTypeForwarders(amap, m, ilModule.GetRawTypeForwarders()) + MemberSignatureEquality = (fun ty1 ty2 -> typeEquivAux EraseAll (globalsOwner.GetTcGlobals()) ty1 ty2) + TypeForwarders = ImportILAssemblyTypeForwarders(amap, m, ilModule.GetRawTypeForwarders()) CSharpStyleExtensionMembersCache = ConcurrentDictionary(1, 0) #if !NO_TYPEPROVIDERS XmlDocumentationInfo = @@ -2434,17 +2432,17 @@ and [] TcImports | None -> () | Some data -> - let fixupThunk () = - data.OptionalFixup(fun nm -> availableToOptionalCcu (tcImports.FindCcu(ctok, m, nm, lookupOnly = false))) - |> ignore + let fixupThunk () = + data.OptionalFixup(fun nm -> availableToOptionalCcu (tcImports.FindCcu(ctok, m, nm, lookupOnly = false))) + |> ignore - fixupThunk () + fixupThunk () - for ccuThunk in data.FixupThunks do - if ccuThunk.IsUnresolvedReference then - tciLock.AcquireLock(fun tcitok -> - RequireTcImportsLock(tcitok, ccuThunks) - ccuThunks.Add(ccuThunk, fixupThunk))) + for ccuThunk in data.FixupThunks do + if ccuThunk.IsUnresolvedReference then + tciLock.AcquireLock(fun tcitok -> + RequireTcImportsLock(tcitok, ccuThunks) + ccuThunks.Add(ccuThunk, fixupThunk))) #if !NO_TYPEPROVIDERS ccuRawDataAndInfos |> List.iter (fun (_, _, phase2) -> phase2 ()) #endif @@ -2571,10 +2569,12 @@ and [] TcImports // A project's own output changes with every build, and phase2 adds provided namespaces into // a type provider assembly's contents - if r.ProjectReference.IsNone - && not isMultiModule - && not (ambiguous.Contains name) - && not (isTypeProviderAssembly data) then + if + r.ProjectReference.IsNone + && not isMultiModule + && not (ambiguous.Contains name) + && not (isTypeProviderAssembly data) + then shareable.Add name |> ignore // A reference must resolve to what the key pins - this batch or the framework layer - or to @@ -2697,9 +2697,11 @@ and [] TcImports // // Only with reduceMemoryUsage: without it TcImports disposal closes a memory-mapped reader, // so a ccu outliving the project that imported it would read from a closed file. - if tcConfig.shareImportedAssemblies - && importsBase.IsSome - && tcConfig.reduceMemoryUsage = ReduceMemoryFlag.Yes then + if + tcConfig.shareImportedAssemblies + && importsBase.IsSome + && tcConfig.reduceMemoryUsage = ReduceMemoryFlag.Yes + then Some(sharedKeys resolved) else None From 09a3cb1dbc832f9e2d4e8711e4726f9043eb91c0 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 19 Aug 2026 14:41:42 +0200 Subject: [PATCH 05/11] Enable by default --- src/Compiler/Driver/CompilerConfig.fs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/Compiler/Driver/CompilerConfig.fs b/src/Compiler/Driver/CompilerConfig.fs index 3e1eac8134a..13d49882c3f 100644 --- a/src/Compiler/Driver/CompilerConfig.fs +++ b/src/Compiler/Driver/CompilerConfig.fs @@ -848,7 +848,7 @@ type TcConfigBuilder = xmlDocInfoLoader = None exiter = QuitProcessExiter parallelReferenceResolution = ParallelReferenceResolution.On - shareImportedAssemblies = false + shareImportedAssemblies = true captureIdentifiersWhenParsing = false typeCheckingConfig = { From 22d58675dca390a846dc779abd2a3c46af16540a Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 26 Aug 2026 10:18:08 +0200 Subject: [PATCH 06/11] Review --- src/Compiler/Driver/CompilerConfig.fs | 16 +- src/Compiler/Driver/CompilerConfig.fsi | 13 +- src/Compiler/Driver/CompilerImports.fs | 322 ++++++++++++-------- src/Compiler/Driver/CompilerImports.fsi | 8 +- src/Compiler/Service/IncrementalBuild.fs | 13 +- src/Compiler/Service/IncrementalBuild.fsi | 3 +- src/Compiler/Service/TransparentCompiler.fs | 3 +- src/Compiler/Service/service.fs | 5 +- src/Compiler/Service/service.fsi | 2 +- 9 files changed, 229 insertions(+), 156 deletions(-) diff --git a/src/Compiler/Driver/CompilerConfig.fs b/src/Compiler/Driver/CompilerConfig.fs index 13d49882c3f..a43b69fd837 100644 --- a/src/Compiler/Driver/CompilerConfig.fs +++ b/src/Compiler/Driver/CompilerConfig.fs @@ -445,6 +445,13 @@ type TypeCheckingConfig = DumpGraph: bool } +[] +type ImportConfig = + { + LangVersion: decimal + CheckNullness: bool + } + [] type TcConfigBuilder = { @@ -645,8 +652,6 @@ type TcConfigBuilder = mutable parallelReferenceResolution: ParallelReferenceResolution - /// Whether the Entity graph imported from a referenced assembly may be shared with other projects - /// that resolve that assembly, and everything it can reach, to the same files mutable shareImportedAssemblies: bool mutable captureIdentifiersWhenParsing: bool @@ -1405,6 +1410,13 @@ type TcConfig private (data: TcConfigBuilder, validate: bool) = member _.shareImportedAssemblies = data.shareImportedAssemblies member _.captureIdentifiersWhenParsing = data.captureIdentifiersWhenParsing member _.typeCheckingConfig = data.typeCheckingConfig + + member _.importConfig = + { + ImportConfig.LangVersion = data.langVersion.SpecifiedVersion + ImportConfig.CheckNullness = data.checkNullness + } + member _.dumpSignatureData = data.dumpSignatureData member _.realsig = data.realsig member _.compilationMode = data.compilationMode diff --git a/src/Compiler/Driver/CompilerConfig.fsi b/src/Compiler/Driver/CompilerConfig.fsi index 60528944861..6d6b39796b8 100644 --- a/src/Compiler/Driver/CompilerConfig.fsi +++ b/src/Compiler/Driver/CompilerConfig.fsi @@ -225,6 +225,13 @@ type TypeCheckingConfig = DumpGraph: bool } +/// The parts of a TcConfig that change how dlls are imported into F# ccus. Every cache of an imported +/// form keys on these, so a new import-affecting setting is added once, here. +[] +type ImportConfig = + { LangVersion: decimal + CheckNullness: bool } + [] type TcConfigBuilder = { @@ -515,8 +522,8 @@ type TcConfigBuilder = mutable parallelReferenceResolution: ParallelReferenceResolution - /// Whether the Entity graph imported from a referenced assembly may be shared with other projects - /// that resolve that assembly, and everything it can reach, to the same files + /// Whether an assembly's imported Entity graph may be shared with projects that resolve it, and + /// everything it can reach, to the same files mutable shareImportedAssemblies: bool mutable captureIdentifiersWhenParsing: bool @@ -898,6 +905,8 @@ type TcConfig = member typeCheckingConfig: TypeCheckingConfig + member importConfig: ImportConfig + member dumpSignatureData: bool member realsig: bool diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index 8ef9d7da214..308a6922df3 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -431,16 +431,50 @@ type AssemblyResolution = this.ilAssemblyRef <- Some assemblyRef assemblyRef -/// Shares the imported form of an assembly between projects. -/// -/// Two projects that resolve an assembly, and everything it can reach, to the same files can share its -/// Entity graph. Its edges point at the CcuThunks of its own closure, so the key covers all of that. module internal SharedImportedCcus = + /// As TcImports keys assemblies: no path, extension or version + type SimpleAssemblyName = string + + /// ILAssemblyRef cannot serve: it does not separate one package's builds for different target + /// frameworks, which differ only in content. + [] + type AssemblyFileId = + | AssemblyFileId of text: string + + member this.Text = + match this with + | AssemblyFileId text -> text + + type AssemblyKeyInfo = + { + File: AssemblyFileId + + /// Its references and its type-forwarder targets, which is all a facade can reach + References: SimpleAssemblyName list + } + + /// Two projects meet at an entry only when every part of this agrees, which is when they would have + /// built the same thing. + [] + type SharedCcuKey = + { + /// Hash over the file each assembly of the closure resolved to + Closure: string + + /// The layer the entry resolves through + FrameworkStamp: int64 + + Config: ImportConfig + + /// Set where an assembly carries several F# ccus + CcuName: SimpleAssemblyName option + } + /// Weak: an entry is held by the projects using it, and holds the rest of its own closure - let private cache = ConcurrentDictionary>() + let private cache = ConcurrentDictionary>() - /// Assembly names are compared case insensitively, as TcImports compares them + /// As TcImports compares them let private nameComparer = StringComparer.OrdinalIgnoreCase /// What a shared ccu may close over besides its own closure. Functions because TcImports comes later. @@ -452,13 +486,13 @@ module internal SharedImportedCcus = } /// What a shared ccu resolves through, in place of the TcImports of whichever project imported it - /// first: the ccus of its own closure, then the framework layer - both pinned by its key. Holding - /// those closure ccus strongly is what lets the cache be weak. - type SharedImportContext(layer: FrameworkLayer) = + /// first: its own closure, then the framework layer - both pinned by its key. Holding those closure + /// ccus strongly is what lets the cache be weak. + type SharedImportContext(layer: FrameworkLayer, closure: SimpleAssemblyName list) = - let refs = ConcurrentDictionary(nameComparer) + let refs = ConcurrentDictionary(nameComparer) - let pending = ResizeArray() + let pending = ResizeArray() let loader = { new AssemblyLoader with @@ -481,23 +515,40 @@ module internal SharedImportedCcus = let importMap = lazy ImportMap(layer.Globals(), loader) - member _.AddClosureRef(name: string, ccu: CcuThunk) = refs[name] <- ccu + member _.Closure = closure + + member _.AddClosureRef(name: SimpleAssemblyName, ccu: CcuThunk) = refs[name] <- ccu member _.GetImportMap() = importMap.Force() /// Publishing before the closure is known would let another project take an entry that resolves /// nothing - member _.HoldForPublication(key: string, ccu: CcuThunk) = pending.Add(key, ccu) + member _.HoldForPublication(key: SharedCcuKey, ccu: CcuThunk) = pending.Add(key, ccu) member _.Pending = pending - let private closureOf (assemblies: Dictionary) root = - let inside = HashSet(nameComparer) - let outside = HashSet(nameComparer) + /// Absent where the assembly is not shareable, and the import stays private to its project + type SharedImport = + { + Key: SharedCcuKey + Context: SharedImportContext + } + + type ShareableAssembly = + { + Key: SharedCcuKey + Closure: SimpleAssemblyName list + } + + /// The names walked to inside the batch, then those leaving it, which are framework assemblies the + /// key pins by stamp + let private closureOf (assemblies: Dictionary) root = + let inside = HashSet(nameComparer) + let outside = HashSet(nameComparer) let rec walk n = if inside.Add n then - for r in snd assemblies[n] do + for r in assemblies[n].References do if assemblies.ContainsKey r then walk r else @@ -506,32 +557,41 @@ module internal SharedImportedCcus = walk root inside, outside - /// A key per shareable assembly, covering its whole closure, so two projects meet at an entry only when - /// everything behind it resolved to the same file too. `assemblies` maps each simple name in the batch - /// to the file it resolved to and the names it can reach; names leaving it are framework assemblies, - /// which the configuration key pins. - /// /// An assembly whose closure is not shareable gets no key: its entities, and the per-CCU caches they /// carry, can point into one project's copy of an unshared assembly. - let computeKeys configKey (assemblies: Dictionary) isShareable = - let keys = Dictionary(nameComparer) + let computeKeys + (frameworkStamp: int64) + (config: ImportConfig) + (assemblies: Dictionary) + isShareable + = + let keys = Dictionary(nameComparer) - for KeyValue(name, _) in assemblies do + for KeyValue(name, assembly) in assemblies do let inside, outside = closureOf assemblies name if Seq.forall isShareable inside then let parts = - Seq.append (inside |> Seq.map (fun n -> n + "|" + fst assemblies[n])) (outside |> Seq.map (fun r -> "unresolved:" + r)) + Seq.append + (inside |> Seq.map (fun n -> n + "|" + assemblies[n].File.Text)) + (outside |> Seq.map (fun r -> "unresolved:" + r)) |> Seq.sort keys[name] <- - Md5StringHasher.empty - |> Md5StringHasher.addString configKey - |> Md5StringHasher.addStrings parts + { + Key = + { + Closure = Md5StringHasher.addStrings parts Md5StringHasher.empty + FrameworkStamp = frameworkStamp + Config = config + CcuName = None + } + Closure = assembly.References + } keys - let private tryGet (key: string) = + let private tryGet (key: SharedCcuKey) = match cache.TryGetValue key with | true, wr -> match wr.TryGetTarget() with @@ -541,22 +601,20 @@ module internal SharedImportedCcus = None | _ -> None - /// The shared entry for an assembly, or a fresh one from `build` held until its closure is known. - /// `build` is not called on a hit, which is what lets the F# path skip unpickling entirely. - let getOrBuild (entry: (string * SharedImportContext) option) (build: unit -> CcuThunk) = + /// `build` is not called on a hit, which is what lets the F# path skip unpickling entirely + let getOrBuild (entry: SharedImport option) (build: unit -> CcuThunk) = match entry with | None -> build () - | Some(key, ctx) -> - match tryGet key with + | Some entry -> + match tryGet entry.Key with | Some ccu -> ccu | None -> let ccu = build () - ctx.HoldForPublication(key, ccu) + entry.Context.HoldForPublication(entry.Key, ccu) ccu - /// Last writer wins: each of two projects importing at once keeps the ccu it built, and both are - /// internally consistent - let add (key: string) (ccu: CcuThunk) = + /// Last writer wins: two projects importing at once each keep the ccu they built, both consistent + let add (key: SharedCcuKey) (ccu: CcuThunk) = cache[key] <- WeakReference ccu let clear () = cache.Clear() @@ -1354,8 +1412,8 @@ and [] TcImports let tciLock = TcImportsLock() - /// Identifies this instance in cache keys: an identity hash is neither unique nor stable across - /// collection, and aliasing two import layers would silently mix their ccus + /// Identifies this instance in cache keys: an identity hash is neither unique nor stable, and + /// aliasing two import layers would silently mix their ccus let stamp = newStamp () //---- Start protected by tciLock ------- @@ -2203,7 +2261,7 @@ and [] TcImports // clear when else it is required, e.g. for Mono. member tcImports.PrepareToImportReferencedILAssembly - (ctok, m, fileName, dllinfo: ImportedBinary, ?shared: string * SharedImportedCcus.SharedImportContext) + (ctok, m, fileName, dllinfo: ImportedBinary, ?shared: SharedImportedCcus.SharedImport) = CheckDisposed() let tcConfig = tcConfigP.Get ctok @@ -2212,10 +2270,10 @@ and [] TcImports let ilScopeRef = dllinfo.ILScopeRef let invalidateCcu = Event<_>() - let sharedContext = shared |> Option.map snd + let sharedContext = shared |> Option.map (fun s -> s.Context) - // Everything the imported ccu later resolves goes through this thunk, so it decides whether the ccu - // is bound to the importing project or to what the sharing key pins + // Everything the ccu later resolves goes through this thunk, so it decides whether the ccu is + // bound to the importing project or to what the key pins let amap = match sharedContext with | Some ctx -> ctx.GetImportMap @@ -2276,13 +2334,13 @@ and [] TcImports phase2 member tcImports.PrepareToImportReferencedFSharpAssembly - (ctok, m, fileName, dllinfo: ImportedBinary, ?shared: string * SharedImportedCcus.SharedImportContext) + (ctok, m, fileName, dllinfo: ImportedBinary, ?shared: SharedImportedCcus.SharedImport) = CheckDisposed() - let sharedContext = shared |> Option.map snd + let sharedContext = shared |> Option.map (fun s -> s.Context) - // GetTcGlobals would reach the same object - the field is only set on the framework layer - but + // GetTcGlobals reaches the same object - the field is only set on the framework layer - but // through a project's TcImports, which a cached ccu must not hold let globalsOwner = tcImports.KeyPinnedLayer @@ -2303,8 +2361,12 @@ and [] TcImports let ccuRawDataAndInfos = ilModule.GetRawFSharpSignatureData(m, ilShortAssemName, fileName) |> List.map (fun (ccuName, (sigDataReader, sigDataReaderB)) -> - // One entry per ccu, because an assembly can carry several - let entry = shared |> Option.map (fun (key, ctx) -> key + "|fsharp|" + ccuName, ctx) + let entry = + shared + |> Option.map (fun s -> + { s with + Key = { s.Key with CcuName = Some ccuName } + }) let optDatas = Map.ofList optDataReaders @@ -2315,51 +2377,53 @@ and [] TcImports // None on a hit: nothing is left to relink, and the signature data is never read let mutable dataOpt = None - let ccu = - SharedImportedCcus.getOrBuild entry (fun () -> - let data = - GetSignatureData(fileName, ilScopeRef, ilModule.TryGetILModuleDef(), sigDataReader, sigDataReaderB) + let importFresh () = + let data = + GetSignatureData(fileName, ilScopeRef, ilModule.TryGetILModuleDef(), sigDataReader, sigDataReaderB) - let minfo: PickledCcuInfo = data.RawData - let mspec = minfo.mspec + let minfo: PickledCcuInfo = data.RawData + let mspec = minfo.mspec - if mspec.DisplayName = "FSharp.Core" then - updateSeqTypeIsPrefix mspec + // Fixes up the unpickled contents, so a hit gets it too + if mspec.DisplayName = "FSharp.Core" then + updateSeqTypeIsPrefix mspec - let codeDir = minfo.compileTimeWorkingDir + let codeDir = minfo.compileTimeWorkingDir - // note: for some fields we fix up this information later - let ccuData: CcuData = - { - ILScopeRef = ilScopeRef - Stamp = newStamp () - FileName = Some fileName - QualifiedName = Some(ilScopeRef.QualifiedName) - SourceCodeDirectory = codeDir - IsFSharp = true - Contents = mspec + // note: for some fields we fix up this information later + let ccuData: CcuData = + { + ILScopeRef = ilScopeRef + Stamp = newStamp () + FileName = Some fileName + QualifiedName = Some(ilScopeRef.QualifiedName) + SourceCodeDirectory = codeDir + IsFSharp = true + Contents = mspec #if !NO_TYPEPROVIDERS - InvalidateEvent = invalidateCcu.Publish - IsProviderGenerated = false - ImportProvidedType = (fun ty -> ImportProvidedType (amap ()) m ty) + InvalidateEvent = invalidateCcu.Publish + IsProviderGenerated = false + ImportProvidedType = (fun ty -> ImportProvidedType (amap ()) m ty) #endif - TryGetILModuleDef = ilModule.TryGetILModuleDef - UsesFSharp20PlusQuotations = minfo.usesQuotations - MemberSignatureEquality = (fun ty1 ty2 -> typeEquivAux EraseAll (globalsOwner.GetTcGlobals()) ty1 ty2) - TypeForwarders = ImportILAssemblyTypeForwarders(amap, m, ilModule.GetRawTypeForwarders()) - CSharpStyleExtensionMembersCache = ConcurrentDictionary(1, 0) + TryGetILModuleDef = ilModule.TryGetILModuleDef + UsesFSharp20PlusQuotations = minfo.usesQuotations + MemberSignatureEquality = (fun ty1 ty2 -> typeEquivAux EraseAll (globalsOwner.GetTcGlobals()) ty1 ty2) + TypeForwarders = ImportILAssemblyTypeForwarders(amap, m, ilModule.GetRawTypeForwarders()) + CSharpStyleExtensionMembersCache = ConcurrentDictionary(1, 0) #if !NO_TYPEPROVIDERS - XmlDocumentationInfo = - match tcConfig.xmlDocInfoLoader with - | Some xmlDocInfoLoader -> xmlDocInfoLoader.TryLoad(fileName) - | _ -> None + XmlDocumentationInfo = + match tcConfig.xmlDocInfoLoader with + | Some xmlDocInfoLoader -> xmlDocInfoLoader.TryLoad(fileName) + | _ -> None #else - XmlDocumentationInfo = None + XmlDocumentationInfo = None #endif - } + } - dataOpt <- Some data - CcuThunk.Create(ccuName, ccuData)) + dataOpt <- Some data + CcuThunk.Create(ccuName, ccuData) + + let ccu = SharedImportedCcus.getOrBuild entry importFresh let optdata = InterruptibleLazy(fun _ -> @@ -2494,11 +2558,9 @@ and [] TcImports return None } - /// Every assembly name a piece of metadata can reach: its own, the ones it references, and the - /// targets of its type forwarders, which is all a facade's closure consists of. Also reports a - /// multi-module assembly, whose auxiliary modules need a loader bound to the importing project. - let reachableAssemblyNames (self: string) (data: IRawFSharpAssemblyData) = - let names = ResizeArray() + /// Also reports a multi-module assembly, whose auxiliary modules need the importing project + let reachableAssemblyNames self (data: IRawFSharpAssemblyData) = + let names = ResizeArray() names.Add self for aref in data.ILAssemblyRefs do @@ -2524,48 +2586,47 @@ and [] TcImports |> List.exists (fun a -> a.Method.DeclaringType.BasicQualifiedName.Contains "TypeProviderAssembly") | None -> false - /// The file an assembly resolved to, as ILModuleReaderCacheKey identifies one. ILAssemblyRef cannot - /// separate one package's builds for different target frameworks, which differ only in content. - let fileIdentity (r: AssemblyResolution) = + let fileIdentity (r: AssemblyResolution) : SharedImportedCcus.AssemblyFileId = let writeStamp = try string (FileSystem.GetLastWriteTimeShim r.resolvedPath).Ticks with _ -> "nostamp" - r.resolvedPath + "|" + writeStamp + SharedImportedCcus.AssemblyFileId(r.resolvedPath + "|" + writeStamp) let sharedKeys (all: (AssemblyResolution * IRawFSharpAssemblyData) list) = let tcConfig = tcConfigP.Get ctok let ic = StringComparer.OrdinalIgnoreCase let short (p: string) = Path.GetFileNameWithoutExtension p - // Anything that changes how types are imported belongs here. The framework layer goes in by - // stamp, since FrameworkImportsCache already shares it per configuration. - let configKey = - sprintf - "%d|%b|%O|%b" - (match importsBase with - | Some b -> b.Stamp - | None -> 0L) - tcConfig.checkNullness - tcConfig.langVersion.SpecifiedVersion - tcConfig.xmlDocInfoLoader.IsSome + // By stamp: FrameworkImportsCache already keys the layer on the same config + let frameworkStamp = + match importsBase with + | Some b -> b.Stamp + | None -> 0L // Two resolutions claiming one name would key whichever was seen last, so neither is shared let ambiguous = all |> List.countBy (fun (r, _) -> short r.resolvedPath) |> List.choose (fun (name, n) -> if n > 1 then Some name else None) - |> fun names -> HashSet(names, ic) + |> fun names -> HashSet<_>(names, ic) + + let assemblies = + Dictionary(ic) - let assemblies = Dictionary(ic) - let shareable = HashSet(ic) + let shareable = HashSet<_>(ic) for r, data in all do let name = short r.resolvedPath let refs, isMultiModule = reachableAssemblyNames name data - assemblies[name] <- fileIdentity r, refs + + assemblies[name] <- + { + File = fileIdentity r + References = refs + } // A project's own output changes with every build, and phase2 adds provided namespaces into // a type provider assembly's contents @@ -2580,7 +2641,7 @@ and [] TcImports // A reference must resolve to what the key pins - this batch or the framework layer - or to // nothing anywhere, which every sharer agrees about. One only an earlier batch of this project // resolves would be keyed as unresolved while this project resolves it. - let pinnedByKey = Dictionary(ic) + let pinnedByKey = Dictionary<_, bool>(ic) let isPinnedByKey ref = match pinnedByKey.TryGetValue ref with @@ -2602,21 +2663,15 @@ and [] TcImports v for name in List.ofSeq shareable do - if not (snd assemblies[name] |> List.forall isPinnedByKey) then + if not (assemblies[name].References |> List.forall isPinnedByKey) then shareable.Remove name |> ignore - // The closure travels with the key: it is what the entry's import context has to resolve - let shared = Dictionary(ic) + SharedImportedCcus.computeKeys frameworkStamp tcConfig.importConfig assemblies shareable.Contains - for KeyValue(name, key) in SharedImportedCcus.computeKeys configKey assemblies shareable.Contains do - shared[name] <- key, snd assemblies[name] - - shared - - let contexts = ResizeArray() + let contexts = ResizeArray() let registerDll - (keys: Dictionary option) + (keys: Dictionary option) (r: AssemblyResolution, assemblyData: IRawFSharpAssemblyData) = let m = r.originalReference.Range @@ -2624,17 +2679,20 @@ and [] TcImports let ilShortAssemName = assemblyData.ShortAssemblyName let ilScopeRef = assemblyData.ILScopeRef - // A project's own output is never shared, and can share a simple name with a package, so it is - // checked here rather than left to the key table + // A project's own output can share a simple name with a package, so it is excluded here + // rather than left to the key table let shared = match keys with | Some keys when r.ProjectReference.IsNone -> match keys.TryGetValue(Path.GetFileNameWithoutExtension fileName) with - | true, (k, refs) -> - let ctx = SharedImportedCcus.SharedImportContext frameworkLayer + | true, shareable -> + let ctx = SharedImportedCcus.SharedImportContext(frameworkLayer, shareable.Closure) + + contexts.Add ctx + + let import: SharedImportedCcus.SharedImport = { Key = shareable.Key; Context = ctx } - contexts.Add(ctx, refs) - Some(k, ctx) + Some import | _ -> None | _ -> None @@ -2712,13 +2770,13 @@ and [] TcImports let! ccuinfos = phase2s |> runMethod - // Every assembly is registered now, so each new entry can be given its closure and published; - // nothing has forced an import yet, so nothing can have resolved through a half-filled one - for ctx, refs in contexts do + // Everything is registered, so each entry can take its closure and be published; no import + // has been forced yet, so nothing can have resolved through a half-filled context + for ctx in contexts do if ctx.Pending.Count > 0 then - for r in refs do - match tcImports.FindCcu(ctok, range0, r, lookupOnly = true) with - | ResolvedCcu ccu -> ctx.AddClosureRef(r, ccu) + for name in ctx.Closure do + match tcImports.FindCcu(ctok, range0, name, lookupOnly = true) with + | ResolvedCcu ccu -> ctx.AddClosureRef(name, ccu) | UnresolvedCcu _ -> () for key, ccu in ctx.Pending do diff --git a/src/Compiler/Driver/CompilerImports.fsi b/src/Compiler/Driver/CompilerImports.fsi index 15ae2b06ec1..e4bdb1c4b40 100644 --- a/src/Compiler/Driver/CompilerImports.fsi +++ b/src/Compiler/Driver/CompilerImports.fsi @@ -98,13 +98,11 @@ type ResolvedExtensionReference = | ResolvedExtensionReference of string * AssemblyReference list * Tainted list #endif -/// Holds the contents imported from a referenced assembly so that projects resolving that assembly, and -/// everything it can reach, to the same files share one copy of its Entity graph. Switched on per checker -/// by tcConfig.shareImportedAssemblies. Entries are weak, so nothing is retained on a project's behalf -/// once that project is gone. +/// Shares one copy of an assembly's Entity graph between the projects resolving that assembly, and +/// everything it can reach, to the same files. Entries are weak: nothing is retained on a project's behalf. module internal SharedImportedCcus = - /// Drops every entry, for FSharpChecker.ClearCaches + /// For FSharpChecker.ClearCaches val clear: unit -> unit /// Represents a resolved imported binary diff --git a/src/Compiler/Service/IncrementalBuild.fs b/src/Compiler/Service/IncrementalBuild.fs index d3c79be7c52..0ff85715d89 100644 --- a/src/Compiler/Service/IncrementalBuild.fs +++ b/src/Compiler/Service/IncrementalBuild.fs @@ -487,15 +487,15 @@ type BoundModel private ( ) /// Global service state -type FrameworkImportsCacheKey = - | FrameworkImportsCacheKey of resolvedpath: string list * assemblyName: string * targetFrameworkDirectories: string list * fsharpBinaries: string * langVersion: decimal * checkNulls: bool +type FrameworkImportsCacheKey = + | FrameworkImportsCacheKey of resolvedpath: string list * assemblyName: string * targetFrameworkDirectories: string list * fsharpBinaries: string * importConfig: ImportConfig interface ICacheKey with member this.GetKey() = - this |> function FrameworkImportsCacheKey(assemblyName=a;checkNulls=c) -> if c then a + "CheckNulls" else a + this |> function FrameworkImportsCacheKey(assemblyName=a;importConfig=c) -> if c.CheckNullness then a + "CheckNulls" else a - member this.GetLabel() = - this |> function FrameworkImportsCacheKey(assemblyName=a;checkNulls=c) -> if c then a + "CheckNulls" else a + member this.GetLabel() = + this |> function FrameworkImportsCacheKey(assemblyName=a;importConfig=c) -> if c.CheckNullness then a + "CheckNulls" else a member this.GetVersion() = this @@ -532,8 +532,7 @@ type FrameworkImportsCache(size) = tcConfig.primaryAssembly.Name, tcConfig.GetTargetFrameworkDirectories(), tcConfig.fsharpBinariesDir, - tcConfig.langVersion.SpecifiedVersion, - tcConfig.checkNullness) + tcConfig.importConfig) let node = lock gate (fun () -> diff --git a/src/Compiler/Service/IncrementalBuild.fsi b/src/Compiler/Service/IncrementalBuild.fsi index fa2b46553d3..c3b4d3398c5 100644 --- a/src/Compiler/Service/IncrementalBuild.fsi +++ b/src/Compiler/Service/IncrementalBuild.fsi @@ -28,8 +28,7 @@ type internal FrameworkImportsCacheKey = assemblyName: string * targetFrameworkDirectories: string list * fsharpBinaries: string * - langVersion: decimal * - checkNulls: bool + importConfig: ImportConfig interface ICacheKey diff --git a/src/Compiler/Service/TransparentCompiler.fs b/src/Compiler/Service/TransparentCompiler.fs index 37f5cfb7c17..dc49c27bbab 100644 --- a/src/Compiler/Service/TransparentCompiler.fs +++ b/src/Compiler/Service/TransparentCompiler.fs @@ -614,8 +614,7 @@ type internal TransparentCompiler tcConfig.primaryAssembly.Name, tcConfig.GetTargetFrameworkDirectories(), tcConfig.fsharpBinariesDir, - tcConfig.langVersion.SpecifiedVersion, - tcConfig.checkNullness + tcConfig.importConfig ) caches.FrameworkImports.Get( diff --git a/src/Compiler/Service/service.fs b/src/Compiler/Service/service.fs index e2d9f6450bd..16d89d8332e 100644 --- a/src/Compiler/Service/service.fs +++ b/src/Compiler/Service/service.fs @@ -219,7 +219,7 @@ type FSharpChecker if keepAssemblyContents && enablePartialTypeChecking then invalidArg "enablePartialTypeChecking" "'keepAssemblyContents' and 'enablePartialTypeChecking' cannot be both enabled." - let shareImportedAssemblies = defaultArg shareImportedAssemblies false + let shareImportedAssemblies = defaultArg shareImportedAssemblies true let parallelReferenceResolution = inferParallelReferenceResolution parallelReferenceResolution @@ -337,8 +337,7 @@ type FSharpChecker braceMatchCache.Clear(utok) backgroundCompiler.ClearCaches() ClearAllILModuleReaderCache() - // Entries are weak, so live projects keep the ones they use and the rest go on their own, but - // "clear the caches" should mean it: after this nothing is held on any project's behalf. + // Entries are weak, but "clear the caches" should mean it FSharp.Compiler.CompilerImports.SharedImportedCcus.clear () member ic.ClearLanguageServiceRootCachesAndCollectAndFinalizeAllTransients() = diff --git a/src/Compiler/Service/service.fsi b/src/Compiler/Service/service.fsi index bfa94b41605..40410c4a004 100644 --- a/src/Compiler/Service/service.fsi +++ b/src/Compiler/Service/service.fsi @@ -32,7 +32,7 @@ type public FSharpChecker = /// Indicates whether a table of symbol keys should be kept for background compilation /// Indicates whether to perform partial type checking. Cannot be set to true if keepAssemblyContents is true. If set to true, can cause duplicate type-checks when richer information on a file is needed, but can skip background type-checking entirely on implementation files with signature files. /// Indicates whether to resolve references in parallel. - /// Default: false. Indicates whether the contents imported from a referenced assembly may be shared between projects that resolve that assembly, and everything it can reach, to the same files. Saves memory and import time in solutions where projects reference the same binaries, and costs nothing where they do not. Assemblies containing type providers, and assemblies produced by a project in the same solution, are never shared. + /// Default: true. Indicates whether the contents imported from a referenced assembly may be shared between projects that resolve that assembly, and everything it can reach, to the same files. Saves memory and import time in solutions where projects reference the same binaries, and costs nothing where they do not. Assemblies containing type providers, and assemblies produced by a project in the same solution, are never shared. Set to false to opt out. /// When set to true we create a set of all identifiers for each parsed file which can be used to speed up finding references. /// Default: FileSystem. You can use Custom source to provide a function that will return the source for a given file path instead of reading it from the file system. Note that with this option the FSharpChecker will also not monitor the file system for file changes. It will expect to be notified of changes via the NotifyFileChanged method. /// Default: false. Indicates whether we use a new experimental background compiler. This does not yet support all features From 869fe65e3fd37e07bcf41734f0198c89a2167ec1 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 26 Aug 2026 10:28:57 +0200 Subject: [PATCH 07/11] Cleanup --- src/Compiler/Driver/CompilerImports.fs | 6 +++++- src/Compiler/Service/IncrementalBuild.fs | 4 ++-- src/Compiler/Service/service.fs | 4 ++-- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index 308a6922df3..80cdec74fa4 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -2579,12 +2579,16 @@ and [] TcImports List.distinct (List.ofSeq names), multiModule +#if NO_TYPEPROVIDERS + let isTypeProviderAssembly (_: IRawFSharpAssemblyData) = false +#else let isTypeProviderAssembly (data: IRawFSharpAssemblyData) = match data.TryGetILModuleDef() with | Some ilModule -> ilModule.ManifestOfAssembly.CustomAttrs.AsList() - |> List.exists (fun a -> a.Method.DeclaringType.BasicQualifiedName.Contains "TypeProviderAssembly") + |> List.exists (TryDecodeTypeProviderAssemblyAttr >> Option.isSome) | None -> false +#endif let fileIdentity (r: AssemblyResolution) : SharedImportedCcus.AssemblyFileId = let writeStamp = diff --git a/src/Compiler/Service/IncrementalBuild.fs b/src/Compiler/Service/IncrementalBuild.fs index 0ff85715d89..b7bf86700d3 100644 --- a/src/Compiler/Service/IncrementalBuild.fs +++ b/src/Compiler/Service/IncrementalBuild.fs @@ -487,14 +487,14 @@ type BoundModel private ( ) /// Global service state -type FrameworkImportsCacheKey = +type FrameworkImportsCacheKey = | FrameworkImportsCacheKey of resolvedpath: string list * assemblyName: string * targetFrameworkDirectories: string list * fsharpBinaries: string * importConfig: ImportConfig interface ICacheKey with member this.GetKey() = this |> function FrameworkImportsCacheKey(assemblyName=a;importConfig=c) -> if c.CheckNullness then a + "CheckNulls" else a - member this.GetLabel() = + member this.GetLabel() = this |> function FrameworkImportsCacheKey(assemblyName=a;importConfig=c) -> if c.CheckNullness then a + "CheckNulls" else a member this.GetVersion() = this diff --git a/src/Compiler/Service/service.fs b/src/Compiler/Service/service.fs index 16d89d8332e..a07f5f90e4e 100644 --- a/src/Compiler/Service/service.fs +++ b/src/Compiler/Service/service.fs @@ -11,6 +11,7 @@ open FSharp.Compiler.Caches open FSharp.Compiler.CodeAnalysis open FSharp.Compiler.CodeAnalysis.TransparentCompiler open FSharp.Compiler.CompilerConfig +open FSharp.Compiler.CompilerImports open FSharp.Compiler.CompilerOptions open FSharp.Compiler.Diagnostics open FSharp.Compiler.Driver @@ -337,8 +338,7 @@ type FSharpChecker braceMatchCache.Clear(utok) backgroundCompiler.ClearCaches() ClearAllILModuleReaderCache() - // Entries are weak, but "clear the caches" should mean it - FSharp.Compiler.CompilerImports.SharedImportedCcus.clear () + SharedImportedCcus.clear () member ic.ClearLanguageServiceRootCachesAndCollectAndFinalizeAllTransients() = use _ = From 09e5d27be65306a79dc214bee33c80ed3646b405 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 26 Aug 2026 10:39:29 +0200 Subject: [PATCH 08/11] Cleanup --- src/Compiler/Driver/CompilerImports.fs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index 80cdec74fa4..643e3f84993 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -462,9 +462,10 @@ module internal SharedImportedCcus = /// Hash over the file each assembly of the closure resolved to Closure: string - /// The layer the entry resolves through + /// The layer the entry resolves through, and so the TcGlobals import reads FrameworkStamp: int64 + /// Implied by FrameworkStamp today - the framework key covers it - but not by construction Config: ImportConfig /// Set where an assembly carries several F# ccus From aa7a9aecfce84311f6a580ad410d3be816ed4068 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 26 Aug 2026 11:06:49 +0200 Subject: [PATCH 09/11] Cleanup --- src/Compiler/Driver/CompilerImports.fs | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index 643e3f84993..a1e5388a48a 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -462,12 +462,9 @@ module internal SharedImportedCcus = /// Hash over the file each assembly of the closure resolved to Closure: string - /// The layer the entry resolves through, and so the TcGlobals import reads + /// The layer the entry resolves through, and so the TcGlobals and ImportConfig import reads FrameworkStamp: int64 - /// Implied by FrameworkStamp today - the framework key covers it - but not by construction - Config: ImportConfig - /// Set where an assembly carries several F# ccus CcuName: SimpleAssemblyName option } @@ -560,12 +557,7 @@ module internal SharedImportedCcus = /// An assembly whose closure is not shareable gets no key: its entities, and the per-CCU caches they /// carry, can point into one project's copy of an unshared assembly. - let computeKeys - (frameworkStamp: int64) - (config: ImportConfig) - (assemblies: Dictionary) - isShareable - = + let computeKeys (frameworkStamp: int64) (assemblies: Dictionary) isShareable = let keys = Dictionary(nameComparer) for KeyValue(name, assembly) in assemblies do @@ -584,7 +576,6 @@ module internal SharedImportedCcus = { Closure = Md5StringHasher.addStrings parts Md5StringHasher.empty FrameworkStamp = frameworkStamp - Config = config CcuName = None } Closure = assembly.References @@ -1560,6 +1551,9 @@ and [] TcImports member _.Stamp = stamp + /// The config this layer's ccus were imported under + member _.GetImportConfig ctok = (tcConfigP.Get ctok).importConfig + /// The layer a sharing key pins by stamp, and so the only one a shared ccu may close over member tcImports.KeyPinnedLayer = match importsBase with @@ -2601,11 +2595,9 @@ and [] TcImports SharedImportedCcus.AssemblyFileId(r.resolvedPath + "|" + writeStamp) let sharedKeys (all: (AssemblyResolution * IRawFSharpAssemblyData) list) = - let tcConfig = tcConfigP.Get ctok let ic = StringComparer.OrdinalIgnoreCase let short (p: string) = Path.GetFileNameWithoutExtension p - // By stamp: FrameworkImportsCache already keys the layer on the same config let frameworkStamp = match importsBase with | Some b -> b.Stamp @@ -2671,7 +2663,7 @@ and [] TcImports if not (assemblies[name].References |> List.forall isPinnedByKey) then shareable.Remove name |> ignore - SharedImportedCcus.computeKeys frameworkStamp tcConfig.importConfig assemblies shareable.Contains + SharedImportedCcus.computeKeys frameworkStamp assemblies shareable.Contains let contexts = ResizeArray() @@ -2760,10 +2752,14 @@ and [] TcImports // // Only with reduceMemoryUsage: without it TcImports disposal closes a memory-mapped reader, // so a ccu outliving the project that imported it would read from a closed file. + // + // The config check is what makes the stamp enough to pin the config: a layer built under + // a different one means the framework key did not separate them, so stop rather than share if tcConfig.shareImportedAssemblies && importsBase.IsSome && tcConfig.reduceMemoryUsage = ReduceMemoryFlag.Yes + && importsBase.Value.GetImportConfig ctok = tcConfig.importConfig then Some(sharedKeys resolved) else From e5e26055c7bada52fe936a663da83f82831b9512 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 26 Aug 2026 11:15:31 +0200 Subject: [PATCH 10/11] Cleanup --- src/Compiler/Driver/CompilerConfig.fs | 8 ++++---- src/Compiler/Driver/CompilerConfig.fsi | 8 ++++---- src/Compiler/Driver/CompilerImports.fs | 6 +++--- src/Compiler/Service/IncrementalBuild.fs | 8 ++++---- src/Compiler/Service/IncrementalBuild.fsi | 2 +- src/Compiler/Service/TransparentCompiler.fs | 2 +- 6 files changed, 17 insertions(+), 17 deletions(-) diff --git a/src/Compiler/Driver/CompilerConfig.fs b/src/Compiler/Driver/CompilerConfig.fs index a43b69fd837..00d1a9acb2f 100644 --- a/src/Compiler/Driver/CompilerConfig.fs +++ b/src/Compiler/Driver/CompilerConfig.fs @@ -446,7 +446,7 @@ type TypeCheckingConfig = } [] -type ImportConfig = +type ImportReuseKey = { LangVersion: decimal CheckNullness: bool @@ -1411,10 +1411,10 @@ type TcConfig private (data: TcConfigBuilder, validate: bool) = member _.captureIdentifiersWhenParsing = data.captureIdentifiersWhenParsing member _.typeCheckingConfig = data.typeCheckingConfig - member _.importConfig = + member _.importReuseKey = { - ImportConfig.LangVersion = data.langVersion.SpecifiedVersion - ImportConfig.CheckNullness = data.checkNullness + ImportReuseKey.LangVersion = data.langVersion.SpecifiedVersion + ImportReuseKey.CheckNullness = data.checkNullness } member _.dumpSignatureData = data.dumpSignatureData diff --git a/src/Compiler/Driver/CompilerConfig.fsi b/src/Compiler/Driver/CompilerConfig.fsi index 6d6b39796b8..4ce3d416e56 100644 --- a/src/Compiler/Driver/CompilerConfig.fsi +++ b/src/Compiler/Driver/CompilerConfig.fsi @@ -225,10 +225,10 @@ type TypeCheckingConfig = DumpGraph: bool } -/// The parts of a TcConfig that change how dlls are imported into F# ccus. Every cache of an imported -/// form keys on these, so a new import-affecting setting is added once, here. +/// A field belongs here when two projects that differ in it cannot reuse one imported form. Every cache +/// of an imported form keys on this, so a new one is added once, here. [] -type ImportConfig = +type ImportReuseKey = { LangVersion: decimal CheckNullness: bool } @@ -905,7 +905,7 @@ type TcConfig = member typeCheckingConfig: TypeCheckingConfig - member importConfig: ImportConfig + member importReuseKey: ImportReuseKey member dumpSignatureData: bool diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index a1e5388a48a..fdc72d107ca 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -462,7 +462,7 @@ module internal SharedImportedCcus = /// Hash over the file each assembly of the closure resolved to Closure: string - /// The layer the entry resolves through, and so the TcGlobals and ImportConfig import reads + /// The layer the entry resolves through, and so the TcGlobals import reads FrameworkStamp: int64 /// Set where an assembly carries several F# ccus @@ -1552,7 +1552,7 @@ and [] TcImports member _.Stamp = stamp /// The config this layer's ccus were imported under - member _.GetImportConfig ctok = (tcConfigP.Get ctok).importConfig + member _.GetImportReuseKey ctok = (tcConfigP.Get ctok).importReuseKey /// The layer a sharing key pins by stamp, and so the only one a shared ccu may close over member tcImports.KeyPinnedLayer = @@ -2759,7 +2759,7 @@ and [] TcImports tcConfig.shareImportedAssemblies && importsBase.IsSome && tcConfig.reduceMemoryUsage = ReduceMemoryFlag.Yes - && importsBase.Value.GetImportConfig ctok = tcConfig.importConfig + && importsBase.Value.GetImportReuseKey ctok = tcConfig.importReuseKey then Some(sharedKeys resolved) else diff --git a/src/Compiler/Service/IncrementalBuild.fs b/src/Compiler/Service/IncrementalBuild.fs index b7bf86700d3..f3dc5086fe4 100644 --- a/src/Compiler/Service/IncrementalBuild.fs +++ b/src/Compiler/Service/IncrementalBuild.fs @@ -488,14 +488,14 @@ type BoundModel private ( /// Global service state type FrameworkImportsCacheKey = - | FrameworkImportsCacheKey of resolvedpath: string list * assemblyName: string * targetFrameworkDirectories: string list * fsharpBinaries: string * importConfig: ImportConfig + | FrameworkImportsCacheKey of resolvedpath: string list * assemblyName: string * targetFrameworkDirectories: string list * fsharpBinaries: string * importReuseKey: ImportReuseKey interface ICacheKey with member this.GetKey() = - this |> function FrameworkImportsCacheKey(assemblyName=a;importConfig=c) -> if c.CheckNullness then a + "CheckNulls" else a + this |> function FrameworkImportsCacheKey(assemblyName=a;importReuseKey=c) -> if c.CheckNullness then a + "CheckNulls" else a member this.GetLabel() = - this |> function FrameworkImportsCacheKey(assemblyName=a;importConfig=c) -> if c.CheckNullness then a + "CheckNulls" else a + this |> function FrameworkImportsCacheKey(assemblyName=a;importReuseKey=c) -> if c.CheckNullness then a + "CheckNulls" else a member this.GetVersion() = this @@ -532,7 +532,7 @@ type FrameworkImportsCache(size) = tcConfig.primaryAssembly.Name, tcConfig.GetTargetFrameworkDirectories(), tcConfig.fsharpBinariesDir, - tcConfig.importConfig) + tcConfig.importReuseKey) let node = lock gate (fun () -> diff --git a/src/Compiler/Service/IncrementalBuild.fsi b/src/Compiler/Service/IncrementalBuild.fsi index c3b4d3398c5..03c37da8216 100644 --- a/src/Compiler/Service/IncrementalBuild.fsi +++ b/src/Compiler/Service/IncrementalBuild.fsi @@ -28,7 +28,7 @@ type internal FrameworkImportsCacheKey = assemblyName: string * targetFrameworkDirectories: string list * fsharpBinaries: string * - importConfig: ImportConfig + importReuseKey: ImportReuseKey interface ICacheKey diff --git a/src/Compiler/Service/TransparentCompiler.fs b/src/Compiler/Service/TransparentCompiler.fs index dc49c27bbab..691dde3e802 100644 --- a/src/Compiler/Service/TransparentCompiler.fs +++ b/src/Compiler/Service/TransparentCompiler.fs @@ -614,7 +614,7 @@ type internal TransparentCompiler tcConfig.primaryAssembly.Name, tcConfig.GetTargetFrameworkDirectories(), tcConfig.fsharpBinariesDir, - tcConfig.importConfig + tcConfig.importReuseKey ) caches.FrameworkImports.Get( From 430d262400c2244114fc7e6ac5f0828462fce469 Mon Sep 17 00:00:00 2001 From: Eugene Auduchinok Date: Wed, 26 Aug 2026 11:18:49 +0200 Subject: [PATCH 11/11] Cleanup --- src/Compiler/Driver/CompilerConfig.fsi | 5 +- src/Compiler/Driver/CompilerImports.fs | 62 ++++++++----------------- src/Compiler/Driver/CompilerImports.fsi | 1 - 3 files changed, 20 insertions(+), 48 deletions(-) diff --git a/src/Compiler/Driver/CompilerConfig.fsi b/src/Compiler/Driver/CompilerConfig.fsi index 4ce3d416e56..5ed1050561c 100644 --- a/src/Compiler/Driver/CompilerConfig.fsi +++ b/src/Compiler/Driver/CompilerConfig.fsi @@ -225,8 +225,7 @@ type TypeCheckingConfig = DumpGraph: bool } -/// A field belongs here when two projects that differ in it cannot reuse one imported form. Every cache -/// of an imported form keys on this, so a new one is added once, here. +/// A field belongs here when two projects differing in it cannot reuse one imported form [] type ImportReuseKey = { LangVersion: decimal @@ -522,8 +521,6 @@ type TcConfigBuilder = mutable parallelReferenceResolution: ParallelReferenceResolution - /// Whether an assembly's imported Entity graph may be shared with projects that resolve it, and - /// everything it can reach, to the same files mutable shareImportedAssemblies: bool mutable captureIdentifiersWhenParsing: bool diff --git a/src/Compiler/Driver/CompilerImports.fs b/src/Compiler/Driver/CompilerImports.fs index fdc72d107ca..e311c56c110 100644 --- a/src/Compiler/Driver/CompilerImports.fs +++ b/src/Compiler/Driver/CompilerImports.fs @@ -433,11 +433,9 @@ type AssemblyResolution = module internal SharedImportedCcus = - /// As TcImports keys assemblies: no path, extension or version type SimpleAssemblyName = string - /// ILAssemblyRef cannot serve: it does not separate one package's builds for different target - /// frameworks, which differ only in content. + /// ILAssemblyRef cannot serve: it does not separate one package's builds for different frameworks [] type AssemblyFileId = | AssemblyFileId of text: string @@ -454,8 +452,6 @@ module internal SharedImportedCcus = References: SimpleAssemblyName list } - /// Two projects meet at an entry only when every part of this agrees, which is when they would have - /// built the same thing. [] type SharedCcuKey = { @@ -472,7 +468,6 @@ module internal SharedImportedCcus = /// Weak: an entry is held by the projects using it, and holds the rest of its own closure let private cache = ConcurrentDictionary>() - /// As TcImports compares them let private nameComparer = StringComparer.OrdinalIgnoreCase /// What a shared ccu may close over besides its own closure. Functions because TcImports comes later. @@ -483,9 +478,8 @@ module internal SharedImportedCcus = XmlDoc: string -> XmlDocumentationInfo option } - /// What a shared ccu resolves through, in place of the TcImports of whichever project imported it - /// first: its own closure, then the framework layer - both pinned by its key. Holding those closure - /// ccus strongly is what lets the cache be weak. + /// What a shared ccu resolves through instead of the TcImports that first imported it: its own + /// closure, then the framework layer. Holding the closure strongly is what lets the cache be weak. type SharedImportContext(layer: FrameworkLayer, closure: SimpleAssemblyName list) = let refs = ConcurrentDictionary(nameComparer) @@ -502,7 +496,6 @@ module internal SharedImportedCcus = member _.TryFindXmlDocumentationInfo assemblyName = layer.XmlDoc assemblyName #if !NO_TYPEPROVIDERS - // Type provider assemblies are never shared, so reaching these means that exclusion broke member _.GetProvidedAssemblyInfo(_ctok, m, _assembly) = error (InternalError("a shared ccu cannot import provided types", m)) @@ -519,13 +512,11 @@ module internal SharedImportedCcus = member _.GetImportMap() = importMap.Force() - /// Publishing before the closure is known would let another project take an entry that resolves - /// nothing + /// Publishing before the closure is known would hand another project an entry resolving nothing member _.HoldForPublication(key: SharedCcuKey, ccu: CcuThunk) = pending.Add(key, ccu) member _.Pending = pending - /// Absent where the assembly is not shareable, and the import stays private to its project type SharedImport = { Key: SharedCcuKey @@ -538,8 +529,7 @@ module internal SharedImportedCcus = Closure: SimpleAssemblyName list } - /// The names walked to inside the batch, then those leaving it, which are framework assemblies the - /// key pins by stamp + /// Names walked to inside the batch, then those leaving it - framework assemblies the stamp pins let private closureOf (assemblies: Dictionary) root = let inside = HashSet(nameComparer) let outside = HashSet(nameComparer) @@ -555,8 +545,8 @@ module internal SharedImportedCcus = walk root inside, outside - /// An assembly whose closure is not shareable gets no key: its entities, and the per-CCU caches they - /// carry, can point into one project's copy of an unshared assembly. + /// No key where the closure is not shareable: entities and their per-CCU caches would point into one + /// project's copy of an unshared assembly. let computeKeys (frameworkStamp: int64) (assemblies: Dictionary) isShareable = let keys = Dictionary(nameComparer) @@ -1404,8 +1394,8 @@ and [] TcImports let tciLock = TcImportsLock() - /// Identifies this instance in cache keys: an identity hash is neither unique nor stable, and - /// aliasing two import layers would silently mix their ccus + /// For cache keys: an identity hash is neither unique nor stable, and aliasing two layers would + /// silently mix their ccus let stamp = newStamp () //---- Start protected by tciLock ------- @@ -1551,10 +1541,8 @@ and [] TcImports member _.Stamp = stamp - /// The config this layer's ccus were imported under member _.GetImportReuseKey ctok = (tcConfigP.Get ctok).importReuseKey - /// The layer a sharing key pins by stamp, and so the only one a shared ccu may close over member tcImports.KeyPinnedLayer = match importsBase with | Some b -> b @@ -2267,21 +2255,18 @@ and [] TcImports let sharedContext = shared |> Option.map (fun s -> s.Context) - // Everything the ccu later resolves goes through this thunk, so it decides whether the ccu is - // bound to the importing project or to what the key pins + // Everything the ccu later resolves goes through this, so it decides what the ccu is bound to let amap = match sharedContext with | Some ctx -> ctx.GetImportMap | None -> tcImports.GetImportMap - // Multi-module assemblies are excluded from sharing, so a shared import can never need this let auxModuleLoader = match sharedContext with | Some _ -> fun scoref -> error (InternalError(sprintf "a shared ccu cannot load the auxiliary module %A" scoref, m)) | None -> tcImports.MkLoaderForMultiModuleILAssemblies ctok m - // Meaningless in a ccu handed to other projects: SymbolHelpers.fileNameOfItem combines it with a - // relative path out of the assembly's metadata + // Meaningless in a shared ccu: SymbolHelpers.fileNameOfItem joins it with a path from metadata let sourceDir = match shared with | Some _ -> "" @@ -2335,8 +2320,7 @@ and [] TcImports let sharedContext = shared |> Option.map (fun s -> s.Context) - // GetTcGlobals reaches the same object - the field is only set on the framework layer - but - // through a project's TcImports, which a cached ccu must not hold + // GetTcGlobals reaches the same object, but through a TcImports a cached ccu must not hold let globalsOwner = tcImports.KeyPinnedLayer let amap = @@ -2486,7 +2470,6 @@ and [] TcImports // Relink ccuRawDataAndInfos |> List.iter (fun (dataOpt, _, _) -> - // A shared ccu was relinked by whichever project unpickled it, against the same closure. match dataOpt with | None -> () | Some data -> @@ -2625,8 +2608,7 @@ and [] TcImports References = refs } - // A project's own output changes with every build, and phase2 adds provided namespaces into - // a type provider assembly's contents + // A project's own output changes every build; phase2 mutates a type provider's contents if r.ProjectReference.IsNone && not isMultiModule @@ -2636,8 +2618,7 @@ and [] TcImports shareable.Add name |> ignore // A reference must resolve to what the key pins - this batch or the framework layer - or to - // nothing anywhere, which every sharer agrees about. One only an earlier batch of this project - // resolves would be keyed as unresolved while this project resolves it. + // nothing anywhere. One only an earlier batch resolves would be keyed as unresolved. let pinnedByKey = Dictionary<_, bool>(ic) let isPinnedByKey ref = @@ -2677,7 +2658,6 @@ and [] TcImports let ilScopeRef = assemblyData.ILScopeRef // A project's own output can share a simple name with a package, so it is excluded here - // rather than left to the key table let shared = match keys with | Some keys when r.ProjectReference.IsNone -> @@ -2746,15 +2726,12 @@ and [] TcImports let resolved = assemblyData |> Seq.choose id |> List.ofSeq let keys = - // A framework base exists only for project layers, which register their whole non-framework - // set at once; BuildFrameworkTcImports registers in three batches, and an entry from an - // early one could not resolve what the later ones bring. + // A framework base exists only for project layers, which register their whole set at once; + // BuildFrameworkTcImports uses three batches, and an early entry could not resolve the rest. // - // Only with reduceMemoryUsage: without it TcImports disposal closes a memory-mapped reader, - // so a ccu outliving the project that imported it would read from a closed file. + // Only with reduceMemoryUsage: otherwise disposal closes the reader a surviving ccu needs. // - // The config check is what makes the stamp enough to pin the config: a layer built under - // a different one means the framework key did not separate them, so stop rather than share + // The config check is what makes the stamp enough to pin the config if tcConfig.shareImportedAssemblies && importsBase.IsSome @@ -2771,8 +2748,7 @@ and [] TcImports let! ccuinfos = phase2s |> runMethod - // Everything is registered, so each entry can take its closure and be published; no import - // has been forced yet, so nothing can have resolved through a half-filled context + // Everything is registered and no import forced yet, so no entry can be taken half-filled for ctx in contexts do if ctx.Pending.Count > 0 then for name in ctx.Closure do diff --git a/src/Compiler/Driver/CompilerImports.fsi b/src/Compiler/Driver/CompilerImports.fsi index e4bdb1c4b40..5cf0f846e25 100644 --- a/src/Compiler/Driver/CompilerImports.fsi +++ b/src/Compiler/Driver/CompilerImports.fsi @@ -102,7 +102,6 @@ type ResolvedExtensionReference = /// everything it can reach, to the same files. Entries are weak: nothing is retained on a project's behalf. module internal SharedImportedCcus = - /// For FSharpChecker.ClearCaches val clear: unit -> unit /// Represents a resolved imported binary