too old resource version，Code:410

object="&Status{ListMeta:ListMeta{SelfLink:,
ResourceVersion:,Continue:,RemainingItemCount:nil,},
Status:Failure,Message:too old resource version: 81
006768 (81242378),Reason:Expired,Details:nil,Code:410,}"

  I encountered... When writing code today too old resourceversion This problem ,

• Reasons for screening

Not specified ResourceVersion, Default empty string .kube-apiserver After receiving a read request of this type , It will be to etcd Send a consensus to read / Linear read request acquisition etcd The latest data of the cluster .

ResourceVersion="0", Assignment string 0.kube-apiserver When such a request is received , It may return data of any resource version number , However, the newer version is preferred . In general, it comes directly from kube-apiserver Get the data returned to the cache client, It is possible to read expired data , It is applicable to scenarios that do not require high data consistency .

ResourceVersion For a non 0 String .kube-apiserver When such a request is received , It will guarantee Cache The latest in ResourceVersion Greater than or equal to your incoming ResourceVersion, And then from Cache Find the resource object you requested in key, Return data to client. The basic principle is kube-apiserver For each core resource （ Such as Pod） One was maintained Cache, adopt etcd Of Watch Mechanism to update in real time Cache. When your Get The request carries non 0 Of ResourceVersion, It waits for the latest... In the cache ResourceVersion Greater than or equal to you Get In the request ResoureVersion, If the conditions are met, start from Cache Query data in , Return to client. If the conditions are not met , It waits at most 3 second , If more than 3 second ,Cache The latest in ResourceVersion Less than Get In the request ResourceVersion, It will return ResourceVersionTooLarge Wrong to client.

etcd Default hold 5 Change records within minutes , Each resource change will update a larger resource version ResourceVersion,ResourceVersion Is a global variable shared by all resource types .

about watch The request for , You can designate a resourceVersion=0 To get 5 Records of any changes within minutes and thereafter , This kind of behavior is very strange , Therefore, it is not recommended to specify 0. You can specify one resourceVersion To get the change records after the resource version , But this resource version is older than 5 Minimum version retained within minutes , Will reply with 410 Status code , If greater than the maximum version , You may have to wait , Until timeout .\

• The trigger condition

Service creation exceeds etcd The cluster is saved by default cache After time , If the service comes in resourceversion Version of is earlier than etcd The minimum version retained within the default save time , It will trigger 410(status code), trigger too old resource version.

• resolvent

Upon receipt of 410 statuscode When , Error handling , again rewatch Again 、 Go straight to the code ：

retry := 0
reWatch:
	ch, err := kubeclient.KubeClient.AppsV1().StatefulSets(sts.ObjectMeta.Namespace).Watch(context.TODO(),
		metav1.SingleObject(metav1.ObjectMeta{Name: sts.ObjectMeta.Name, ResourceVersion: resourceVersion}))
	if err != nil {
		if apierrors.IsGone(err) {
			tlog.Error(err, "watch resource is gone", "err is:", sts.Name)
			resourceVersion = ""
			retry++
			if retry < 3 {
				goto reWatch
			}
		}

		tlog.Error(err, "cannot watch resource:%v", "err is:", sts.Name)
		return
	}