On Tue, 9 Oct 2012, Peter Zijlstra wrote:

> Well the code they were patching is in the wakeup path. As I think Tang
> said, we leave !runnable tasks on whatever cpu they ran on last, even if
> that cpu is offlined, we try and fix up state when we get a wakeup.
> 
> On wakeup, it tries to find a cpu to run on and will try a cpu of the
> same node first.
> 
> Now if that node's entirely gone away, it appears the cpu_to_node() map
> will not return a valid node number.
> 
> I think that's a change in behaviour, it didn't used to do that afaik.
> Certainly this code hasn't change in a while.
> 

If cpu_to_node() always returns a valid node id even if all cpus on the 
node are offline, then the cpumask_of_node() implementation, which the 
sched code is using, should either return an empty cpumask (if 
node_to_cpumask_map[nid] isn't freed) or cpu_online_mask.  The change in 
behavior here occurred because 
cpu_hotplug-unmap-cpu2node-when-the-cpu-is-hotremoved.patch in -mm doesn't 
return a valid node id and forces it to return -1 so a kzalloc_node(..., 
-1) fallsback to allocate anywhere.

But if you only need cpu_to_node() when waking up to find a runnable cpu 
for this NUMA information, then I think you can just change the 
kzalloc_node() in alloc_{fair,rt}_sched_group() to do 
kzalloc(..., cpu_online(cpu) ? cpu_to_node(cpu) : NUMA_NO_NODE).

 [ The changelog here is confusing because it's fixing a problem in 
   linux-next without saying so. ]