Kubernetes schedules Pods on the worker nodes based on multiple factors. For the most part, you would want to leave Kubernetes alone to do its job. However, there are cases where you would want to have a say on the scheduling. A great didactic scenario is a mix of nodes having SSDs and HDs (spinning disks) and you would want to influence scheduling by making sure I/O hungry Pods go to nodes with SSDs.
Here is a list I came up with, to illustrate what Kubernetes has to offer. Of course, I take an analogy of people liking/disliking beverages. So, people are Pods and Kubernetes puts people on to the pantries where the beverages are available/not available (nodes).
Case 1: Hard Yes – Strictly coffee only – no other beverage
Node affinity (requiredDuringSchedulingIgnoredDuringExecution). Node selectors can be used as well but will be deprecated in future. Scheduler will schedule the Pod only on coffee nodes.
Case 2: Soft Yes – Coffee is preferred but anything OK if coffee is not available
Node affinity (preferredDuringSchedulingIgnoredDuringExecution). Scheduler will try to schedule the Pod on a coffee node, but will schedule it on tea and soda nodes, if a coffee node is not available.
Case 3: Hard No – Allergic to coffee but anything else is OK
Coffee nodes tainted and Pods do not have tolerance to coffee. Taint effect: NoSchedule. Scheduler will schedule the Pod only on tea and soda nodes.
Case 4: Soft No – Dislike coffee but if nothing else is available, coffee is okay
Coffee nodes tainted and Pods do not have tolerance to coffee. Taint effect: PreferNoSchedule. Scheduler will try to avoid scheduling the Pod to the coffee node, but will schedule it on a coffee node, if no tea and soda nodes are available.
Case 5: Just give me something – Anything is okay
Pods with tolerance to coffee. Scheduler will schedule the Pod anywhere including coffee nodes.
So, that is for node affinity and taint-tolerance.
Now, two additional cases with Pod affinity.
Case 6: Wants to be with other coffee drinkers. Okay, if others are around
Pod affinity with coffee Pods.
Case 7: Does not want to be with tea and soda drinkers
Pod anti-affinity with tea and soda Pods.
Please share your comments, especially if there is a mistake.