本文将讲述如何使用scheduler-framework扩展原生调度器
目的： 在prefilter阶段检查pod是否添加有dely注释，如果未达到对应时间则不调度

分析需要实现的method

注册插件

WithPlugin返回一个注册选项，由此我们可以看出，我们的插件需要实现framework.PluginFactory 接口

func WithPlugin(name string, factory framework.PluginFactory) Option {    return func(registry framework.Registry) error {        return registry.Register(name, factory)    }}

该接口接收传入的附加参数和FrameworkHandle，关于FrameworkHandle的作用请查看之前文章

type PluginFactory = func(configuration *runtime.Unknown, f FrameworkHandle) (Plugin, error)

PreFilterPlugin接口

// PreFilterPlugin is an interface that must be implemented by "prefilter" plugins.// These plugins are called at the beginning of the scheduling cycle.type PreFilterPlugin interface {    Plugin    // PreFilter is called at the beginning of the scheduling cycle. All PreFilter    // plugins must return success or the pod will be rejected.    PreFilter(ctx context.Context, state *CycleState, p *v1.Pod) *Status    // PreFilterExtensions returns a PreFilterExtensions interface if the plugin implements one,    // or nil if it does not. A Pre-filter plugin can provide extensions to incrementally    // modify its pre-processed info. The framework guarantees that the extensions    // AddPod/RemovePod will only be called after PreFilter, possibly on a cloned    // CycleState, and may call those functions more than once before calling    // Filter again on a specific node.    PreFilterExtensions() PreFilterExtensions}// Plugin is the parent type for all the scheduling framework plugins.type Plugin interface {    Name() string}

PreFilterExtensions()方法返回PreFilterExtensions接口

type PreFilterExtensions interface {    // AddPod is called by the framework while trying to evaluate the impact    // of adding podToAdd to the node while scheduling podToSchedule.    AddPod(ctx context.Context, state *CycleState, podToSchedule *v1.Pod, podToAdd *v1.Pod, nodeInfo *schedulernodeinfo.NodeInfo) *Status    // RemovePod is called by the framework while trying to evaluate the impact    // of removing podToRemove from the node while scheduling podToSchedule.    RemovePod(ctx context.Context, state *CycleState, podToSchedule *v1.Pod, podToRemove *v1.Pod, nodeInfo *schedulernodeinfo.NodeInfo) *Status}

我们需要实现五个method:

• Name 返回插件名称
• PreFilter 对pod进行筛选
• PreFilterExtensions prefilter扩展功能，评估add/removepod的影响，如果不实现可返回nil
• AddPod 评估添加pod到node的影响
• RemovePod 评估删除pod到node的影响

代码实现

实现注册相关

const Name = "test"var _ framework.PreFilterPlugin = &TestPlugin{}type Args struct {    KubeConfig string `json:"kubeconfig,omitempty"`    Master     string `json:"master,omitempty"`}type TestPlugin struct {    handle framework.FrameworkHandle    Args   *Args}func New(rargs *runtime.Unknown, handle framework.FrameworkHandle) (framework.Plugin, error) {    args := &Args{}    if err := framework.DecodeInto(rargs, args); err != nil {        return nil, err    }    klog.Info(args)    return &TestPlugin{        handle: handle,        Args:   args,    }, nil}

实现prefilter接口

# 返回名称，任何plugin都需要实现func (self *TestPlugin) Name() string {    return Name}# 实现PreFilter methodfunc (self *TestPlugin) PreFilter(ctx context.Context, state *framework.CycleState, p *v1.Pod) *framework.Status {    klog.Error("into controller test")    state.Write()    var dtime int64    var err error# 判断是否有延迟字段    if v, ok := p.Annotations["delay"]; ok {        if dtime, err = strconv.ParseInt(v, 10, 64); err != nil {            return nil        }        # 距离当前大于延时间，则调度        if time.Now().Unix()-p.CreationTimestamp.Unix() >= dtime {            klog.Infof("scheduler: %s/%s", p.Namespace, p.Name)            return nil        }        # 否则不调度        klog.Infof("not reatch scheduler time: %s/%s", p.Namespace, p.Name)        return framework.NewStatus(framework.Skip, "not reatch scheduler time")    }    return nil}# PreFilterExtensions AddPod method,返回nil即successfunc (self *TestPlugin) AddPod(ctx context.Context, state *framework.CycleState, podToSchedule *v1.Pod, podToAdd *v1.Pod, nodeInfo *schedulernodeinfo.NodeInfo) *framework.Status {    return nil}# PreFilterExtensions RemovePod,返回nil即successfunc (self *TestPlugin) RemovePod(ctx context.Context, state *framework.CycleState, podToSchedule *v1.Pod, podToRemove *v1.Pod, nodeInfo *schedulernodeinfo.NodeInfo) *framework.Status {    return nil}# 这里也可以返回nilfunc (self *TestPlugin) PreFilterExtensions() framework.PreFilterExtensions {    return self}

测试

• 测试时使用kubeadm，停止原有kubelet

# 将static pod配置移走，kubelet会自动停止mv /etc/kubernetes/manifests/kube-scheduler.yaml ./

• 编译运行

go build ./test-scheduler-framework --config=config.yaml

• 测试pod
  创建deploy 配置如下 apiVersion: apps/v1 kind: Deployment metadata: labels: run: busybox name: busybox namespace: default spec: replicas: 0 selector: matchLabels: run: busybox template: metadata: annotations: delay: "15" labels: run: busybox spec: containers: - args: - sleep - "123456" image: busybox name: busybox
• 扩容观察pod状态

$ kubectl scale deploy busybox --replicas=1$ kubectl get pods -l run=busybox -wNAME                      READY   STATUS    RESTARTS   AGEbusybox-8d8554fc8-f9899   0/1     Pending   0          3sbusybox-8d8554fc8-f9899   0/1     Pending   0          85sbusybox-8d8554fc8-f9899   0/1     ContainerCreating   0          85sbusybox-8d8554fc8-f9899   0/1     ContainerCreating   0          86sbusybox-8d8554fc8-f9899   1/1     Running             0          90s

• 观察调度器日志

E1226 18:19:11.071899   92593 type.go:59] into controller testI1226 18:19:11.071909   92593 type.go:70] not reatch scheduler time: default/busybox-8d8554fc8-f9899E1226 18:19:11.071923   92593 framework.go:287] error while running "test" prefilter plugin for pod "busybox-8d8554fc8-f9899": not reatch scheduler timeE1226 18:19:11.071941   92593 factory.go:469] Error scheduling default/busybox-8d8554fc8-f9899: error while running "test" prefilter plugin for pod "busybox-8d8554fc8-f9899": not reatch scheduler time; retryingE1226 18:19:11.071970   92593 scheduler.go:638] error selecting node for pod: error while running "test" prefilter plugin for pod "busybox-8d8554fc8-f9899": not reatch scheduler timeE1226 18:20:36.064233   92593 type.go:59] into controller testI1226 18:20:36.064258   92593 type.go:67] scheduler: default/busybox-8d8554fc8-f9899

可以看到确实延时调度了，但是因为重新调度本身有时间间隔(30s)，所以并不是我们设置的值

​