Get/set global R session pool capacity (DEPRECATED)
Description
Deprecated. Use polars_options() to get, and pl$set_options() to set.
Usage
pl_get_global_rpool_cap()
pl_set_global_rpool_cap(n)
Arguments
n
|
Integer, the capacity limit R sessions to process R code. |
Details
Background R sessions communicate via polars arrow IPC (series/vectors)
or R serialize + shared memory buffers via the rust crate
ipc-channel. Multi-process communication has overhead
because all data must be serialized/de-serialized and sent via buffers.
Using multiple R sessions will likely only give a speed-up in a
low io - high cpu scenario.
Native polars query syntax runs in threads and have no overhead. Polars
has as default double as many thread workers as cores. If any worker are
queuing for or using R sessions, other workers can still continue any
native polars parts as much as possible.
Value
polars_options()$rpool_cap returns the capacity ("limit")
of co-running external R sessions / processes.
polars_options()$rpool_active is the number of R sessions
are already spawned in the pool. rpool_cap is the limit of
new R sessions to spawn. Anytime a polars thread worker needs a
background R session specifically to run R code embedded in a query via
$map_batches(…, in_background = TRUE) or
$map_elements(…, in_background = TRUE), it will obtain any
R session idling in rpool, or spawn a new R session (process) if
capacity is not already reached. If capacity
is already reached, the thread worker will sleep and in a R job queue
until an R session is idle.
Examples
#> [1] 4
#> [1] 8
#> [1] 4