标签
PostgreSQL , 10.0 , 并行计算 , 索引扫描 , 子查询 , VACUUM , 外部表并行
背景
PostgreSQL 9.6推出的多核并行计算特性，支持全表扫描，hash join，聚合操作。
10.0 在此基础上，增加了更多的支持。
1. Parallel bitmap heap scan
2. Parallel Index Scans
3. Parallel Merge Join
4. parallelize queries containing subplans
5. Block level parallel vacuum
6. Extending the parallelism for index-only scans
7. ParallelFinish Hook of FDW/CSP
这是一个fdw钩子，用于在访问FDW/CSP的node（backend process）的内存上下文释放前，让上面的gather node获得上下文的控制权。
从而，从DSM中获得每个fdw node通道的统计信息，比如pg_strom项目，custom scan阶段的dma数据传输的速度，GPU的运算时间等。
使用这个钩子，就可以达到以上目的。
Hello,
The attached patch implements the suggestion by Amit before.
What I'm motivated is to collect extra run-time statistics specific
to a particular ForeignScan/CustomScan, not only the standard
Instrumentation; like DMA transfer rate or execution time of GPU
kernels in my case.
Per-node DSM toc is one of the best way to return run-time statistics
to the master backend, because FDW/CSP can assign arbitrary length of
the region according to its needs. It is quite easy to require.
However, one problem is, the per-node DSM toc is already released when
ExecEndNode() is called on the child node of Gather.
This patch allows extensions to get control on the master backend's
context when all the worker node gets finished but prior to release
of the DSM segment. If FDW/CSP has its special statistics on the
segment, it can move to the private memory area for EXPLAIN output
or something other purpose.
One design consideration is whether the hook shall be called from
ExecParallelRetrieveInstrumentation() or ExecParallelFinish().
The former is a function to retrieve the standard Instrumentation
information, thus, it is valid only if EXPLAIN ANALYZE.
On the other hands, if we put entrypoint at ExecParallelFinish(),
extension can get control regardless of EXPLAIN ANALYZE, however,
it also needs an extra planstate_tree_walker().
Right now, we don't assume anything onto the requirement by FDW/CSP.
It may want run-time statistics regardless of EXPLAIN ANALYZE, thus,
hook shall be invoked always when Gather node confirmed termination
of the worker processes.
Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>
这个patch的讨论，详见邮件组，本文末尾URL。
PostgreSQL社区的作风非常严谨，一个patch可能在邮件组中讨论几个月甚至几年，根据大家的意见反复的修正，patch合并到master已经非常成熟，所以PostgreSQL的稳定性也是远近闻名的。
参考
https://commitfest.postgresql.org/13/812/
https://commitfest.postgresql.org/13/849/
https://commitfest.postgresql.org/13/918/
https://commitfest.postgresql.org/13/941/
https://commitfest.postgresql.org/13/954/
