scheduler crashes occasionally

[Wang-Kai]

What happened:

The volcano scheduler crashes occasionally, which due to IgnoredDevicesList slice isn’t concurrency secure.

What you expected to happen:

Don't panic

How to reproduce it (as minimally and precisely as possible):

The demo like this:

package main

import (
	"strings"
	"time"
)

func main() {
	var vals []string
	go func() {
		for {
			for _, v := range vals {
				strings.Compare(v, "a")
			}
		}
	}()

	go func() {
		for {
			vals = []string{}
			vals = append(vals, "a", "b")
		}
	}()
	time.Sleep(time.Hour)
}

Anything else we need to know?:

Environment:

• Volcano Version: v1.8.2
• Kubernetes version (use kubectl version): v1.20
• Cloud provider or hardware configuration: 128C 256G
• OS (e.g. from /etc/os-release): Debian GNU/Linux 9
• Kernel (e.g. uname -a):Linux 5.10.0
• Install tools: kubectl
• Others:

[lowang-bh]

It will be fixed in #3325