@@ -224,11 +224,25 @@ This module defines the following functions:
224224 Initialize a new (idle) Python interpreter
225225 and return a :class: `Interpreter ` object for it.
226226
227- .. function :: create_queue()
227+ .. function :: create_queue(maxsize=0, *, unbounditems=UNBOUND )
228228
229229 Initialize a new cross-interpreter queue and return a :class: `Queue `
230230 object for it.
231231
232+ *maxsize * sets the upper bound on the number of items that can be placed
233+ in the queue. If *maxsize * is less than or equal to zero, the queue
234+ size is infinite.
235+
236+ *unbounditems * sets the default behavior when getting an item from the
237+ queue whose original interpreter has been destroyed.
238+ See :meth: `Queue.put ` for supported values.
239+
240+ .. function :: is_shareable(obj)
241+
242+ Return ``True `` if the object can be sent to another interpreter
243+ without using :mod: `pickle `, and ``False `` otherwise.
244+ See :ref: `interp-object-sharing `.
245+
232246
233247Interpreter objects
234248^^^^^^^^^^^^^^^^^^^
@@ -261,27 +275,71 @@ Interpreter objects
261275
262276 Finalize and destroy the interpreter.
263277
264- .. method :: prepare_main(ns=None, **kwargs)
278+ .. method :: prepare_main(ns=None, /, **kwargs)
279+
280+ Bind the given objects into the interpreter's :mod: `!__main__ `
281+ module namespace. This is the primary way to pass data to code
282+ running in another interpreter.
283+
284+ *ns * is an optional :class: `dict ` mapping names to values.
285+ Any additional keyword arguments are also bound as names.
286+
287+ The values must be shareable between interpreters. Some objects
288+ are actually shared, some are copied efficiently, and most are
289+ copied via :mod: `pickle `. See :ref: `interp-object-sharing `.
265290
266- Bind objects in the interpreter's :mod: ` !__main__ ` module.
291+ For example::
267292
268- Some objects are actually shared and some are copied efficiently,
269- but most are copied via :mod: `pickle `. See :ref: `interp-object-sharing `.
293+ interp = interpreters.create()
294+ interp.prepare_main(name='world')
295+ interp.exec('print(f"Hello, {name}!")')
270296
271- .. method :: exec(code, /, dedent=True)
297+ This is equivalent to setting variables in the interpreter's
298+ :mod: `!__main__ ` module before calling :meth: `exec ` or :meth: `call `.
299+ The names are available as global variables in the executed code.
300+
301+ .. method :: exec(code, /)
272302
273303 Run the given source code in the interpreter (in the current thread).
274304
305+ *code * is a :class: `str ` of Python source code. It is executed as
306+ though it were the body of a script, using the interpreter's
307+ :mod: `!__main__ ` module as the globals namespace.
308+
309+ There is no return value. To get a result back, use :meth: `call `
310+ instead, or communicate through a :class: `Queue `.
311+
312+ If the code raises an unhandled exception, an :exc: `ExecutionFailed `
313+ exception is raised in the calling interpreter. The actual
314+ exception object is not preserved because objects cannot be shared
315+ between interpreters directly.
316+
317+ This blocks the current thread until the code finishes.
318+
275319 .. method :: call(callable, /, *args, **kwargs)
276320
277- Return the result of calling running the given function in the
278- interpreter (in the current thread).
321+ Call *callable * in the interpreter (in the current thread) and
322+ return the result.
323+
324+ Nearly all callables, args, kwargs, and return values are supported.
325+ All "shareable" objects are supported, as are "stateless" functions
326+ (meaning non-closures that do not use any globals). For other
327+ objects, this method falls back to :mod: `pickle `.
328+
329+ If the callable raises an exception, an :exc: `ExecutionFailed `
330+ exception is raised in the calling interpreter.
279331
280332 .. _interp-call-in-thread :
281333
282334 .. method :: call_in_thread(callable, /, *args, **kwargs)
283335
284- Run the given function in the interpreter (in a new thread).
336+ Start a new :class: `~threading.Thread ` that calls *callable * in
337+ the interpreter and return the thread object.
338+
339+ This is a convenience wrapper that combines
340+ :mod: `threading ` with :meth: `call `. The thread is started
341+ immediately. Call :meth: `~threading.Thread.join ` on the returned
342+ thread to wait for it to finish.
285343
286344Exceptions
287345^^^^^^^^^^
@@ -318,19 +376,86 @@ Communicating Between Interpreters
318376
319377.. class :: Queue(id)
320378
321- A wrapper around a low-level, cross-interpreter queue, which
322- implements the :class: `queue.Queue ` interface. The underlying queue
323- can only be created through :func: `create_queue `.
379+ A cross-interpreter queue that can be used to pass data safely
380+ between interpreters. It provides the same interface as
381+ :class: `queue.Queue `. The underlying queue can only be created
382+ through :func: `create_queue `.
383+
384+ When an object is placed in the queue, it is prepared for use in
385+ another interpreter. Some objects are actually shared and some are
386+ copied efficiently, but most are copied via :mod: `pickle `.
387+ See :ref: `interp-object-sharing `.
324388
325- Some objects are actually shared and some are copied efficiently,
326- but most are copied via :mod: `pickle `. See :ref: `interp-object-sharing `.
389+ :class: `Queue ` objects themselves are shareable between interpreters
390+ (they reference the same underlying queue), making them suitable for
391+ use with :meth: `Interpreter.prepare_main `.
327392
328393 .. attribute :: id
329394
330395 (read-only)
331396
332397 The queue's ID.
333398
399+ .. attribute :: maxsize
400+
401+ (read-only)
402+
403+ The maximum number of items allowed in the queue. A value of zero
404+ means the queue size is infinite.
405+
406+ .. method :: empty()
407+
408+ Return ``True `` if the queue is empty, ``False `` otherwise.
409+
410+ .. method :: full()
411+
412+ Return ``True `` if the queue is full, ``False `` otherwise.
413+
414+ .. method :: qsize()
415+
416+ Return the number of items in the queue.
417+
418+ .. method :: put(obj, block=True, timeout=None, *, unbounditems=None)
419+
420+ Put *obj * into the queue. If *block * is true (the default),
421+ block if necessary until a free slot is available. If *timeout *
422+ is a positive number, block at most *timeout * seconds and raise
423+ :exc: `QueueFullError ` if no free slot is available within that time.
424+
425+ If *block * is false, put *obj * in the queue if a free slot is
426+ immediately available, otherwise raise :exc: `QueueFullError `.
427+
428+ *unbounditems * controls what happens when the item is retrieved
429+ via :meth: `get ` after the interpreter that called :meth: `put ` has
430+ been destroyed. If ``None `` (the default), the queue's default
431+ (set via :func: `create_queue `) is used. Supported values:
432+
433+ * :data: `UNBOUND ` -- :meth: `get ` returns the :data: `UNBOUND `
434+ sentinel in place of the original object.
435+ * :data: `UNBOUND_ERROR ` -- :meth: `get ` raises
436+ :exc: `ItemInterpreterDestroyed `.
437+ * :data: `UNBOUND_REMOVE ` -- the item is silently removed from
438+ the queue when the original interpreter is destroyed.
439+
440+ .. method :: put_nowait(obj, *, unbounditems=None)
441+
442+ Equivalent to ``put(obj, block=False) ``.
443+
444+ .. method :: get(block=True, timeout=None)
445+
446+ Remove and return an item from the queue. If *block * is true
447+ (the default), block if necessary until an item is available.
448+ If *timeout * is a positive number, block at most *timeout * seconds
449+ and raise :exc: `QueueEmptyError ` if no item is available within
450+ that time.
451+
452+ If *block * is false, return an item if one is immediately
453+ available, otherwise raise :exc: `QueueEmptyError `.
454+
455+ .. method :: get_nowait()
456+
457+ Equivalent to ``get(block=False) ``.
458+
334459
335460.. exception :: QueueEmptyError
336461
@@ -354,31 +479,75 @@ Creating an interpreter and running code in it::
354479
355480 interp = interpreters.create()
356481
357- # Run in the current OS thread.
482+ # Run source code directly.
483+ interp.exec('print("Hello from a subinterpreter!")')
358484
359- interp.exec('print("spam!")')
485+ # Call a function and get the result.
486+ def add(x, y):
487+ return x + y
360488
361- interp.exec("""if True:
362- print('spam!')
363- """)
489+ result = interp.call(add, 3, 4)
490+ print(result) # 7
364491
365- from textwrap import dedent
366- interp.exec(dedent("""
367- print('spam!')
368- """))
492+ # Run a function in a new thread.
493+ def worker():
494+ print('Running in a thread!')
369495
370- def run(arg):
371- return arg
496+ t = interp.call_in_thread(worker)
497+ t.join()
372498
373- res = interp.call(run, 'spam!')
374- print(res)
499+ Passing data with :meth: `~Interpreter.prepare_main `::
375500
376- def run():
377- print('spam!')
501+ interp = interpreters.create()
378502
379- interp.call(run)
503+ # Bind variables into the interpreter's __main__ namespace.
504+ interp.prepare_main(greeting='Hello', name='world')
505+ interp.exec('print(f"{greeting}, {name}!")')
380506
381- # Run in new OS thread.
507+ # Can also use a dict.
508+ config = {'host': 'localhost', 'port': 8080}
509+ interp.prepare_main(config)
510+ interp.exec('print(f"Connecting to {host}:{port}")')
382511
383- t = interp.call_in_thread(run)
384- t.join()
512+ Using queues to communicate between interpreters::
513+
514+ interp = interpreters.create()
515+
516+ # Create a queue and share it with the subinterpreter.
517+ queue = interpreters.create_queue()
518+ interp.prepare_main(queue=queue)
519+
520+ # The subinterpreter puts results into the queue.
521+ interp.exec("""
522+ import math
523+ queue.put(math.factorial(10))
524+ """)
525+
526+ # The main interpreter reads from the same queue.
527+ result = queue.get()
528+ print(result) # 3628800
529+
530+ Running CPU-bound work in parallel using threads and interpreters::
531+
532+ import time
533+ from concurrent import interpreters
534+
535+ def compute(n):
536+ total = sum(range(n))
537+ return total
538+
539+ interp1 = interpreters.create()
540+ interp2 = interpreters.create()
541+
542+ # Each interpreter runs in its own thread and does not share
543+ # the GIL, enabling true parallel execution.
544+ t1 = interp1.call_in_thread(compute, 50_000_000)
545+ t2 = interp2.call_in_thread(compute, 50_000_000)
546+ t1.join()
547+ t2.join()
548+
549+ .. tip ::
550+
551+ For many use cases, :class: `~concurrent.futures.InterpreterPoolExecutor `
552+ provides a higher-level interface that combines threads with
553+ interpreters automatically.
0 commit comments