INFO:root:start with arguments Namespace(batch_size=128, benchmark=0, data_nthreads=4, data_train='oss://tfmnist-mxnet/mxnet-ext-data/cifar10_train.rec', data_val='oss://tfmnist-mxnet/mxnet-ext-data/cifar10_val.rec', disp_batches=20, gpus='0', image_shape='3,28,28', kv_store='local', load_epoch=None, lr=0.1, lr_factor=0.1, lr_step_epochs='200,250', max_random_aspect_ratio=0, max_random_h=36, max_random_l=50, max_random_rotate_angle=0, max_random_s=50, max_random_scale=1, max_random_shear_ratio=0, min_random_scale=1, model_prefix='oss://tfmnist-mxnet/mxnet-ext-model/', mom=0.9, network='resnet', num_classes=10, num_epochs=1, num_examples=50000, num_layers=50, optimizer='sgd', pad_size=4, random_crop=1, random_mirror=1, rgb_mean='123.68,116.779,103.939', test_io=0, top_k=0, wd=0.0001)
[14:06:53] /home/xuchen/pai-mxnet/src/io/iter_image_recordio.cc:221: ImageRecordIOParser: oss://tfmnist-mxnet/mxnet-ext-data/cifar10_train.rec, use 3 threads for decoding..
[14:06:53] /home/xuchen/pai-mxnet/dmlc-core/include/dmlc/logging.h:300: [14:06:53] /home/xuchen/pai-mxnet/dmlc-core/src/io/input_split_base.cc:163: Check failed: files_.size() != 0U (0 vs. 0) Cannot find any files that matches the URI patternz oss://tfmnist-mxnet/mxnet-ext-data/cifar10_train.rec
Stack trace returned 30 entries:
[bt] (0) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x26) [0x7fda64329d66]
[bt] (1) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc2io14InputSplitBase17InitInputFileInfoERKSs+0x1bd9) [0x7fda65332329]
[bt] (2) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc2io14InputSplitBase4InitEPNS0_10FileSystemEPKcm+0x49) [0x7fda65333329]
[bt] (3) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc10InputSplit6CreateEPKcjjS2_+0x8e4) [0x7fda6531df64]
[bt] (4) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io19ImageRecordIOParserIfE4InitERKSt6vectorISt4pairISsSsESaIS5_EE+0x39a) [0x7fda643c167a]
[bt] (5) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io15ImageRecordIterIfE4InitERKSt6vectorISt4pairISsSsESaIS5_EE+0x89) [0x7fda643c1ab9]
[bt] (6) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io18ImageNormalizeIter4InitERKSt6vectorISt4pairISsSsESaIS4_EE+0x87) [0x7fda643be907]
[bt] (7) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io11BatchLoader4InitERKSt6vectorISt4pairISsSsESaIS4_EE+0x187) [0x7fda643ae697]
[bt] (8) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io14PrefetcherIter4InitERKSt6vectorISt4pairISsSsESaIS4_EE+0x103) [0x7fda643adfe3]
[bt] (9) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(MXDataIterCreateIter+0x1d0) [0x7fda64327750]
[bt] (10) /lib64/libffi.so.6(ffi_call_unix64+0x4c) [0x7fda8170adac]
[bt] (11) /lib64/libffi.so.6(ffi_call+0x1f5) [0x7fda8170a6d5]
[bt] (12) /usr/lib64/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x30b) [0x7fda8191dc8b]
[bt] (13) /usr/lib64/python2.7/lib-dynload/_ctypes.so(+0xaa85) [0x7fda81917a85]
[bt] (14) /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fda8a2d00b3]
[bt] (15) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x1d4c) [0x7fda8a36425c]
[bt] (16) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (17) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x425f) [0x7fda8a36676f]
[bt] (18) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (19) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x425f) [0x7fda8a36676f]
[bt] (20) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (21) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x425f) [0x7fda8a36676f]
[bt] (22) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (23) /lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32) [0x7fda8a3681c2]
[bt] (24) /lib64/libpython2.7.so.1.0(+0xfb5ff) [0x7fda8a3815ff]
[bt] (25) /lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x7e) [0x7fda8a3827be]
[bt] (26) /lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xe9) [0x7fda8a383a49]
[bt] (27) /lib64/libpython2.7.so.1.0(Py_Main+0xc9f) [0x7fda8a394b9f]
[bt] (28) /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fda895c1b15]
[bt] (29) python() [0x400721]
Traceback (most recent call last):
File "mxnet_jobs/train_cifar10.py", line 74, in <module>
fit.fit(args, sym, data.get_rec_iter)
File "/worker/mxnet_jobs/common/fit.py", line 101, in fit
(train, val) = data_loader(args, kv)
File "/worker/mxnet_jobs/common/data.py", line 125, in get_rec_iter
part_index = rank)
File "/usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/io.py", line 661, in creator
ctypes.byref(iter_handle)))
File "/usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/base.py", line 123, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [14:06:53] /home/xuchen/pai-mxnet/dmlc-core/src/io/input_split_base.cc:163: Check failed: files_.size() != 0U (0 vs. 0) Cannot find any files that matches the URI patternz oss://tfmnist-mxnet/mxnet-ext-data/cifar10_train.rec
Stack trace returned 30 entries:
[bt] (0) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x26) [0x7fda64329d66]
[bt] (1) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc2io14InputSplitBase17InitInputFileInfoERKSs+0x1bd9) [0x7fda65332329]
[bt] (2) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc2io14InputSplitBase4InitEPNS0_10FileSystemEPKcm+0x49) [0x7fda65333329]
[bt] (3) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc10InputSplit6CreateEPKcjjS2_+0x8e4) [0x7fda6531df64]
[bt] (4) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io19ImageRecordIOParserIfE4InitERKSt6vectorISt4pairISsSsESaIS5_EE+0x39a) [0x7fda643c167a]
[bt] (5) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io15ImageRecordIterIfE4InitERKSt6vectorISt4pairISsSsESaIS5_EE+0x89) [0x7fda643c1ab9]
[bt] (6) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io18ImageNormalizeIter4InitERKSt6vectorISt4pairISsSsESaIS4_EE+0x87) [0x7fda643be907]
[bt] (7) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io11BatchLoader4InitERKSt6vectorISt4pairISsSsESaIS4_EE+0x187) [0x7fda643ae697]
[bt] (8) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet2io14PrefetcherIter4InitERKSt6vectorISt4pairISsSsESaIS4_EE+0x103) [0x7fda643adfe3]
[bt] (9) /usr/lib/python2.7/site-packages/mxnet-0.9.2-py2.7.egg/mxnet/libmxnet.so(MXDataIterCreateIter+0x1d0) [0x7fda64327750]
[bt] (10) /lib64/libffi.so.6(ffi_call_unix64+0x4c) [0x7fda8170adac]
[bt] (11) /lib64/libffi.so.6(ffi_call+0x1f5) [0x7fda8170a6d5]
[bt] (12) /usr/lib64/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x30b) [0x7fda8191dc8b]
[bt] (13) /usr/lib64/python2.7/lib-dynload/_ctypes.so(+0xaa85) [0x7fda81917a85]
[bt] (14) /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fda8a2d00b3]
[bt] (15) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x1d4c) [0x7fda8a36425c]
[bt] (16) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (17) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x425f) [0x7fda8a36676f]
[bt] (18) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (19) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x425f) [0x7fda8a36676f]
[bt] (20) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (21) /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x425f) [0x7fda8a36676f]
[bt] (22) /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fda8a3680bd]
[bt] (23) /lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32) [0x7fda8a3681c2]
[bt] (24) /lib64/libpython2.7.so.1.0(+0xfb5ff) [0x7fda8a3815ff]
[bt] (25) /lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x7e) [0x7fda8a3827be]
[bt] (26) /lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xe9) [0x7fda8a383a49]
[bt] (27) /lib64/libpython2.7.so.1.0(Py_Main+0xc9f) [0x7fda8a394b9f]
[bt] (28) /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fda895c1b15]
[bt] (29) python() [0x400721]
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。