HI,
I have a query which when run utilizes 100% of CPU load.
So, I changed the variable as below.
set global resource_governor_cpu_limit_mode = hard;
CREATE RESOURCE POOL test_pool WITh hard_CPU_Limit_Percentage = 50;
ALTER USER 'root' SET DEFAULT RESOURCE POOL = test_pool;
SET resource_pool = test_pool;
It has been lowered to about 70%.
1) Can you tell me why it’s higher than the set value?
2) Is there a way to make it 50% closer?
Are you running on our cloud service or self-hosted? And how many cores or VCPUs do you have? It may round to the nearest whole VCPU. But that doesn’t explain it for me because core count is almost always divisible by 2, so it should be possible to hit 50% almost exactly.
How did you measure that the system was taking 70% CPU?
It was confirmed that it was working.
At first, 70% CPU were checked in the studio.
When I checked with the ‘TOP 1’ command, it looked normal.
I’m not sure I understand. Also, are you running self-hosted or in cloud?
I run self-hosted in oracle cloud.
I thought it was cpu by ‘extractors’.
Additionally, is the resource pool managed by cgroup or nice?
singlestore> show resource pools;
+-----------------------+-------------------+---------------+-----------------+---------------------------+---------------------------+-----------------+
| Pool_Name | Memory_Percentage | Query_Timeout | Max_Concurrency | Soft_CPU_Limit_Percentage | Hard_CPU_Limit_Percentage | Max_Queue_Depth |
+-----------------------+-------------------+---------------+-----------------+---------------------------+---------------------------+-----------------+
| default_pool | 100 | NULL | NULL | NULL | NULL | NULL |
| system_auto | 100 | NULL | NULL | NULL | NULL | NULL |
| system_optimizer_pool | 100 | NULL | NULL | NULL | 5 | NULL |
| r2 | 100 | NULL | NULL | NULL | 50 | NULL |
+-----------------------+-------------------+---------------+-----------------+---------------------------+---------------------------+-----------------+
CREATE PIPELINE pl_cdr AS
LOAD DATA LINK oci_obj_demo 'demo/cdr_*/*.csv.gz'
BATCH_INTERVAL 10
resource pool r2
INTO TABLE cdr_tbl (id, cdr_type, from_ctn, to_ctn, start_dt, finish_dt, system_id)
FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';
Oh, okay. Right now CPU Resource governance does not work as expected in some cases on cloud platforms because it is not CGroup aware. We are designing a fix for this issue.
