8000 [vm/ffi] `Pointer.asTypedList` shared across isolates causes use after free · Issue #55800 · dart-lang/sdk · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[vm/ffi] Pointer.asTypedList shared across isolates causes use after free #55800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dcharkes opened this issue May 21, 2024 · 5 comments
Open
Assignees
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi triaged Issue has been triaged by sub team

Comments

@dcharkes
Copy link
Contributor
dcharkes commented May 21, 2024

The native finalizers in asTypedList are bound to an isolate, not an isolate group:

sdk/sdk/lib/ffi/ffi.dart

Lines 368 to 373 in 4dd6ee6

external Int8List asTypedList(
int length, {
@Since('3.1') Pointer<NativeFinalizerFunction>? finalizer,
@Since('3.1') Pointer<Void>? token,
});
}

final _asTypedListFinalizer = _NativeFinalizer(_asTypedListFinalizerCallback);

Hypothesis: The TypedData we create out of Pointer is marked unmodifiable.

We need to either mark the typed data as mutable so that the view does not consider the typed data as unmodifiable, and we copy instead of share the object.
Or, we need to attach finalizers in the isolate group instead of the isolate.

Context:

TODO: verify hypothesis. (Filing issue so that I don't forget.)

@dcharkes dcharkes added area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi labels May 21, 2024
@dcharkes dcharkes self-assigned this May 21, 2024
@dcharkes dcharkes changed the title [ffi] Pointer.asTypedList shared across isolates causes use after free [vm/ffi] Pointer.asTypedList shared across isolates causes use after free May 21, 2024
@a-siva a-siva added the triaged Issue has been triaged by sub team label May 29, 2024
@HosseinYousefi
Copy link
Member

Can we prioritize this?

< 8000 !-- '"` -->

@dcharkes
Copy link
Contributor Author
dcharkes commented Mar 3, 2025

Indeed, the typed-data is not copied correctly.

The following example prints flakily all kinds of things.

Output: [0, 0, 0, 0, 26832, 7656, 1, 0, 3, 0].

Repro:

// Copyright (c) 2025, the Dart project authors.  Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

// SharedObjects=ffi_test_functions

import 'dart:async';
import 'dart:ffi';
import 'dart:isolate';

import 'package:expect/expect.dart';

import 'dylib_utils.dart';

void main() {
  // Force dlopen so @Native lookups in DynamicLibrary.process() succeed.
  dlopenGlobalPlatformSpecific('ffi_test_functions');

  test();
}

Future<void> test() async {
  const length = 10;
  final typedList = await Isolate.run(() {
    final ptr = calloc(length, sizeOf<Int16>()).cast<Int16>();
    final typedList = ptr.asTypedList(length, finalizer: freePointer);
    Timer(Duration(milliseconds: 1), () {
      for (int i = 0; i < length; i++) {
        // overwrite initialized zeros
        typedList[0] = i;
      }
    });
    return typedList;
  });
  for (int i = 0; i < length; i++) {
    Expect.equals(typedList[0], 0);
  }
  final completer = Completer<void>();
  Timer(Duration(milliseconds: 1), () {
    completer.complete();
  });
  for (int i = 0; i < length; i++) {
    Expect.equals(typedList[0], 0);
  }
  await completer.future;
  print(typedList);
}

@Native<Pointer<Void> Function(IntPtr num, IntPtr size)>(isLeaf: true)
external Pointer<Void> calloc(int num, int size);

final freePointer = DynamicLibrary.process()
    .lookup<NativeFunction<Void Function(Pointer<Void>)>>('free');

Edit, more specifically, the error is in Isolate.exit & handling of native finalizers

// Copyright (c) 2025, the Dart project authors.  Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

// SharedObjects=ffi_test_functions

import 'dart:async';
import 'dart:ffi';
import 'dart:isolate';
import 'dart:typed_data';

import 'package:expect/expect.dart';

import 'dylib_utils.dart';

void main() {
  // Force dlopen so @Native lookups in DynamicLibrary.process() succeed.
  dlopenGlobalPlatformSpecific('ffi_test_functions');

  test();
}

Future<void> test() async {
  const length = 10;

  var result = Completer<Int16List>();
  final resultPort = RawReceivePort();
  resultPort.handler = (dynamic response) {
    resultPort.close();
    result.complete(response as Int16List);
  };
  unawaited(
    Isolate.spawn((SendPort sendPort) async {
      final ptr = calloc(length, sizeOf<Int16>()).cast<Int16>();
      final typedList = ptr.asTypedList(length, finalizer: freePointer);
      Isolate.exit(sendPort, typedList); // Repros.
      // sendPort.send(typedList); // Doesn't repro.
    }, resultPort.sendPort),
  );
  final typedList = await result.future;
  print(typedList);
  for (int i = 0; i < length; i++) {
    Expect.equals(typedList[i], 0);
  }
}

@Native<Pointer<Void> Function(IntPtr num, IntPtr size)>(isLeaf: true)
external Pointer<Void> calloc(int num, int size);

final freePointer = DynamicLibrary.process()
    .lookup<NativeFunction<Void Function(Pointer<Void>)>>('free');

@dcharkes
Copy link
Contributor Author
dcharkes commented Mar 3, 2025

TL;DR: Attaching NativeFinalizers to objects not implementing Finalizable breaks assumptions in Isolate.exit.

I've tracked down two issues:

1. Isolate.exit shares TypedData without sharing attached NativeFinalizer

Isolate.exit(sendPort, typedList) uses a validation pass to see if the objects can be sent to the other isolate. This validation pass prevents Finalizables from being in the object graph, but it does not prevent ExternalTypedDatas from being in the object graph. No objects are ever copied on this path.

sendPort.send(typedList) does not have issues because it will copy the external typed data. And after that the isolate will shut down and the original typed data is freed and its finalizer runs.

We could potentially fix this by (A) during the finalization pass finding the NativeFinalizer from the TypedData and migrating it to the target isolate near runtime/lib/isolate.cc::MessageValidator. We'd also need to instantiate a new native finalizer and add it to the root set of the target isolate.

(We need to make sure we can do this in a reasonable time complexity from the VM. The VM does not have access to the relevant finalizer entries attached to an object, so it would have to loop through all entries in the VM. A possible better approach would be to have the validate function fill a list of typed-datas, and then in the run-finalizers-eager method consume that list and pass in a target isolate.)

Alternatively, we could fix it by (B) changing the finalizer to be a finalizers from the dart_api.h instead (as originally proposed in #50507 (comment)). This would technically be a breaking change, see next point.

2. We eagerly run NativeFinalizers before the Isolate.exit to guarantee finalizers have been run

sdk/runtime/vm/isolate.cc

Lines 2619 to 2632 in e8c8500

// Ensure native finalizers are run before isolate has shutdown message is
// sent. This way users can rely on the exit message that an isolate will not
// run any Dart code anymore _and_ will not run any native finalizers anymore.
RunAndCleanupFinalizersOnShutdown();
// Post message before LowLevelShutdown that sends onExit message.
// This ensures that exit message comes last.
if (bequest_ != nullptr) {
auto beneficiary = bequest_->beneficiary();
auto handle = bequest_->TakeHandle();
PortMap::PostMessage(
Message::New(beneficiary, handle, Message::kNormalPriority));
bequest_.reset();
}

For (A) we would need to change those semantics to native finalizers are run eagerly, except for the ones attached to objects being send on exit.

For (B) we don't need to change anything.

Solutions

A. Migrating NativeFinalizers on exit. (However, we decided against it in #55050 for the general case.)
B. Use isolate-group C API finalizers.

Option B is technically a breaking change, as someone could be using asTypedList with a finalizer, and relying on the fact that if they don't send the object to another isolate with Isolate.exit the native finalizer is guaranteed to run.

Proposal:

  1. Change the semantics of asTypedList finalizers to be isolate group scoped.
  2. Breaking change announcement.
  3. (In some future: make them isolate scoped with [vm/ffi] Introduce pragma's 'vm:shareable' and 'vm:shared' and make NativeFinalizers live in an isolate group instead of isolate. #55062)

cc @mraleph @mkustermann

@mraleph
Copy link
Member
mraleph commented Mar 3, 2025

TL;DR: Attaching NativeFinalizers to objects not implementing Finalizable breaks assumptions in Isolate.exit.

We should actually tighten typing of NativeFinalizer implementation because you can currently bypass Finalizable requirement by doing a dynamic call. I also see that there are comments in _NativeFinalizer which are not correct, e.g. this one.

We could potentially fix this by (A) during the finalization pass finding the NativeFinalizer from the TypedData and migrating it to the target isolate near runtime/lib/isolate.cc::MessageValidator. We'd also need to instantiate a new native finalizer and add it to the root set of the target isolate.

You could probably do that reasonably fast by collecting all encountered external TypedData in MessageValidator and then checking against these when running native finalizers... So it would not add too much. However it adds unpleasant amount of complexity.

I think the change you propose is okay. asTypedList does not promise finalization on isolate shutdown - so we don't break any promises.

@dcharkes
Copy link
Contributor Author

I realized we implemented this already before, but did not land it: https://dart-review.googlesource.com/c/sdk/+/229544/11. I think we can reuse that implementation with a few modifications. (Rough sketch, I might overlook something.)

  • _detach in Dart is an Expando on the detach key to finalizer ID (unchanged)
    • We omit pruning this map, assuming users will not use some long lived object that outlives the Finalizable as detachkey.
    • The detach method needs to do a runtime entry to cancel the native finalizer
  • We don't need _attached, because we're running native functions as finalizers, not Dart functions.
    • So, we also don't need the port.
  • The ActiveFinalizers in the VM maps finalizer IDs to finalizable handles.
    • This data structure has locking in place because it's VM global. (It would not require locking if it were isolate-local, the GC cannot run concurrently with isolate code. It would require locking it was isolate-group local due to possible concurrency between isolates.)
    • Use @Native leaf calls instead of native entries (faster)
    • The attach, detach, and finalizer runs modify this data structure.
  • Note: FinalizerIds are used to avoid having FinalizableHandle* as unique identifiers, because those addresses can be reused.

@goderbauer goderbauer self-assigned this Apr 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi triaged Issue has been triaged by sub team
Projects
None yet
Development

No branches or pull requests

5 participants
0