kubectl cp: data lost?
1
My file locally:
ls -al spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 16:59 spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
I used kubectl cp to copy this file to centOS pod in k8s, but I got
# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 110690304 Jan 2 22:15 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
but 137448102 != 110690304. Any data lost?
Thanks
UPDATE
I did another kubectl cp, and put more details below:
On MAC:
me$ ls -al target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 17:24 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
me$ md5 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
MD5 (target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar) = ff6ae687b19c97871e65db6e4ea6002f
on CentOS POD of k8s:
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 132054528 Jan 2 22:47 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# md5sum examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
edc4ddc8f011e5b8e79741e15ac1ac1d examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
kubernetes centos kubectl kubernetes-pod asked Jan 2 at 22:23
BAEBAE
2,89563176
|
show 5 more comments
1
My file locally:
ls -al spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 16:59 spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
I used kubectl cp to copy this file to centOS pod in k8s, but I got
# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 110690304 Jan 2 22:15 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
but 137448102 != 110690304. Any data lost?
Thanks
UPDATE
I did another kubectl cp, and put more details below:
On MAC:
me$ ls -al target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 17:24 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
me$ md5 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
MD5 (target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar) = ff6ae687b19c97871e65db6e4ea6002f
on CentOS POD of k8s:
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 132054528 Jan 2 22:47 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# md5sum examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
edc4ddc8f011e5b8e79741e15ac1ac1d examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
asked Jan 2 at 22:23
BAEBAE
2,89563176
Why don't you usesha256sumormd5sumto verify integrity of the file? We're not seeing your cluster on Stack Overflow so we can't tell whether any data is lost or not. It's likely that the file size is pretty different.
– AhmetB - Google
Jan 2 at 22:36
@AhmetAlpBalkan-Google I updated my post
– BAE
Jan 2 at 22:52
isexamples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jaralready in the pod prior to thekubectl cpcommand?
– Rico
Jan 2 at 22:57
1
This works for me for a 200m filekubectlandkube-apiserver1.13.0
– Rico
Jan 3 at 0:44
1
If you have a Java jar file, usually you'd build that into a custom image and reference that from a pod spec in a deployment spec; you wouldn't manuallykubectl cpit around.
– David Maze
Jan 3 at 2:03
|
show 5 more comments
1
1
1
My file locally:
ls -al spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 16:59 spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
I used kubectl cp to copy this file to centOS pod in k8s, but I got
# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 110690304 Jan 2 22:15 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
but 137448102 != 110690304. Any data lost?
Thanks
UPDATE
I did another kubectl cp, and put more details below:
On MAC:
me$ ls -al target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 17:24 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
me$ md5 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
MD5 (target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar) = ff6ae687b19c97871e65db6e4ea6002f
on CentOS POD of k8s:
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 132054528 Jan 2 22:47 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# md5sum examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
edc4ddc8f011e5b8e79741e15ac1ac1d examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
asked Jan 2 at 22:23
BAEBAE
2,89563176
My file locally:
ls -al spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 16:59 spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
I used kubectl cp to copy this file to centOS pod in k8s, but I got
# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 110690304 Jan 2 22:15 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
but 137448102 != 110690304. Any data lost?
Thanks
UPDATE
I did another kubectl cp, and put more details below:
On MAC:
me$ ls -al target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 me 110502799 137448102 Jan 2 17:24 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
me$ md5 target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
MD5 (target/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar) = ff6ae687b19c97871e65db6e4ea6002f
on CentOS POD of k8s:
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# ls -al examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
-rw------- 1 root root 132054528 Jan 2 22:47 examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
root@myspark-master-5d6656bd84-79745:/usr/spark-2.3.1# md5sum examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
edc4ddc8f011e5b8e79741e15ac1ac1d examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar
asked Jan 2 at 22:23
BAEBAE
2,89563176
asked Jan 2 at 22:23
BAEBAE
2,89563176
edited Jan 2 at 22:51
BAE
asked Jan 2 at 22:23
BAEBAE
2,89563176
asked Jan 2 at 22:23
BAEBAE
2,89563176
asked Jan 2 at 22:23
BAEBAE
2,89563176
2,89563176
Why don't you usesha256sumormd5sumto verify integrity of the file? We're not seeing your cluster on Stack Overflow so we can't tell whether any data is lost or not. It's likely that the file size is pretty different.
– AhmetB - Google
Jan 2 at 22:36
@AhmetAlpBalkan-Google I updated my post
– BAE
Jan 2 at 22:52
isexamples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jaralready in the pod prior to thekubectl cpcommand?
– Rico
Jan 2 at 22:57
1
This works for me for a 200m filekubectlandkube-apiserver1.13.0
– Rico
Jan 3 at 0:44
1
If you have a Java jar file, usually you'd build that into a custom image and reference that from a pod spec in a deployment spec; you wouldn't manuallykubectl cpit around.
– David Maze
Jan 3 at 2:03
|
show 5 more comments
Why don't you usesha256sumormd5sumto verify integrity of the file? We're not seeing your cluster on Stack Overflow so we can't tell whether any data is lost or not. It's likely that the file size is pretty different.
– AhmetB - Google
Jan 2 at 22:36
@AhmetAlpBalkan-Google I updated my post
– BAE
Jan 2 at 22:52
isexamples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jaralready in the pod prior to thekubectl cpcommand?
– Rico
Jan 2 at 22:57
1
This works for me for a 200m filekubectlandkube-apiserver1.13.0
– Rico
Jan 3 at 0:44
1
If you have a Java jar file, usually you'd build that into a custom image and reference that from a pod spec in a deployment spec; you wouldn't manuallykubectl cpit around.
– David Maze
Jan 3 at 2:03
Why don't you use
sha256sum or md5sum to verify integrity of the file? We're not seeing your cluster on Stack Overflow so we can't tell whether any data is lost or not. It's likely that the file size is pretty different.– AhmetB - Google
Jan 2 at 22:36
Why don't you use
sha256sum or md5sum to verify integrity of the file? We're not seeing your cluster on Stack Overflow so we can't tell whether any data is lost or not. It's likely that the file size is pretty different.– AhmetB - Google
Jan 2 at 22:36
@AhmetAlpBalkan-Google I updated my post
– BAE
Jan 2 at 22:52
@AhmetAlpBalkan-Google I updated my post
– BAE
Jan 2 at 22:52
is
examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar already in the pod prior to the kubectl cp command?– Rico
Jan 2 at 22:57
is
examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jar already in the pod prior to the kubectl cp command?– Rico
Jan 2 at 22:57
1
1
This works for me for a 200m file
kubectl and kube-apiserver 1.13.0– Rico
Jan 3 at 0:44
This works for me for a 200m file
kubectl and kube-apiserver 1.13.0– Rico
Jan 3 at 0:44
1
1
If you have a Java jar file, usually you'd build that into a custom image and reference that from a pod spec in a deployment spec; you wouldn't manually
kubectl cp it around.– David Maze
Jan 3 at 2:03
If you have a Java jar file, usually you'd build that into a custom image and reference that from a pod spec in a deployment spec; you wouldn't manually
kubectl cp it around.– David Maze
Jan 3 at 2:03
|
show 5 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54013945%2fkubectl-cp-data-lost%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54013945%2fkubectl-cp-data-lost%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Why don't you use
sha256sumormd5sumto verify integrity of the file? We're not seeing your cluster on Stack Overflow so we can't tell whether any data is lost or not. It's likely that the file size is pretty different.– AhmetB - Google
Jan 2 at 22:36
@AhmetAlpBalkan-Google I updated my post
– BAE
Jan 2 at 22:52
is
examples/jars/spark-streaming-1.0-SNAPSHOT-jar-with-dependencies.jaralready in the pod prior to thekubectl cpcommand?– Rico
Jan 2 at 22:57
1
This works for me for a 200m file
kubectlandkube-apiserver1.13.0– Rico
Jan 3 at 0:44
1
If you have a Java jar file, usually you'd build that into a custom image and reference that from a pod spec in a deployment spec; you wouldn't manually
kubectl cpit around.– David Maze
Jan 3 at 2:03