这个在机器学习PAI里config设置了啊，为啥还报错？
问题1：INFO:tensorflow:Calling model_fn. [2023-05-12 16:22:02,669][INFO] Calling model_fn. [2023-05-12 16:22:02,670][INFO] shared embeddings[num=0] Traceback (most recent call last): File "../easy_rec/python/train_eval.py", line 165, in args.check_mode) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/main.py", line 333, in _train_and_evaluate_impl estimator_train.train_and_evaluate(estimator, train_spec, eval_spec) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/compat/estimator_train.py", line 84, in train_and_evaluate result = executor.run() File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/training.py", line 613, in run return self.run_local() File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/training.py", line 714, in run_local saving_listeners=saving_listeners) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/model/easy_rec_estimator.py", line 92, in train max_steps, saving_listeners) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 370, in train loss = self._train_model(input_fn, hooks, saving_listeners) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1161, in _train_model return self._train_model_default(input_fn, hooks, saving_listeners) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1191, in _train_model_default features, labels, ModeKeys.TRAIN, self.config) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1149, in _call_model_fn model_fn_results = self._model_fn(features=features, **kwargs) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/model/easy_rec_estimator.py", line 600, in _model_fn return self._train_model_fn(features, labels, config) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/model/easy_rec_estimator.py", line 131, in _train_model_fn is_training=True) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/model/deepfm.py", line 34, in init self._wide_features, _ = self._input_layer(self._feature_dict, 'wide') File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/layers/input_layer.py", line 96, in call features, group_name, feature_name_to_output_tensors) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/layers/input_layer.py", line 165, in single_call_input_layer self._fc_parser) File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/feature_column/feature_group.py", line 34, in select_columns wide_columns = [fc.wide_columns[x] for x in self._config.feature_names] File "/data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/feature_column/feature_group.py", line 34, in wide_columns = [fc.wide_columns[x] for x in self._config.feature_names] KeyError: 'title'这个在机器学习PAI里config设置了啊，为啥还报错？
问题2：train_input_path: "/data/xulianming/EasyRec/data/trainV2.csv" eval_input_path: "/data/xulianming/EasyRec/data/testV2.csv" model_dir: "/data/xulianming/EasyRec/conf/DeepFM/_ckpt/"
train_config { log_step_count_steps: 500 # fine_tune_checkpoint: "" optimizer_config: { adam_optimizer: { learning_rate: { exponential_decay_learning_rate { initial_learning_rate: 0.0001 decay_steps: 500 decay_factor: 0.5 min_learning_rate: 0.0000001 } } } use_moving_average: false }
sync_replicas: true save_checkpoints_steps: 1000 num_steps: 2000 }
eval_config { metrics_set: { auc {} } }
data_config { separator: "," input_fields: { input_name: "user_id" input_type: STRING default_val:"" } input_fields: { input_name: "office_id" input_type: STRING default_val:"" } input_fields: { input_name: "user_name" input_type: STRING default_val:"" } input_fields: { input_name: "ip" input_type: STRING default_val:"" } input_fields: { input_name: "device" input_type: STRING default_val:"" } input_fields: { input_name: "province" input_type: STRING default_val:"" } input_fields: { input_name: "city" input_type: STRING default_val:"" } input_fields: { input_name: "browser" input_type: STRING default_val:"" } input_fields: { input_name: "network" input_type: STRING default_val:"" } input_fields: { input_name: "os" input_type: STRING default_val:"" } input_fields: { input_name: "version" input_type: STRING default_val:"" } input_fields: { input_name: "timestamp" input_type: STRING default_val:"" } input_fields: { input_name: "item_id" input_type: STRING default_val:"" } input_fields: { input_name: "block" input_type: STRING default_val:"" } input_fields: { input_name: "ctr" input_type: INT64 default_val:"0" } input_fields: { input_name: "title" input_type: STRING default_val:"" } input_fields: { input_name: "query" input_type: STRING default_val:"" } label_fields: "ctr"
batch_size: 8192 prefetch_size: 64 input_type: CSVInput }
feature_config: { features: { input_names: "user_id" feature_type: IdFeature hash_bucket_size: 1000000 embedding_dim: 16 } features: { input_names: "office_id" feature_type: IdFeature hash_bucket_size: 4 embedding_dim: 16 } features: { input_names: "user_name" feature_type: IdFeature hash_bucket_size: 10 embedding_dim: 16 } features: { input_names: "ip" feature_type: IdFeature embedding_dim: 16 hash_bucket_size: 50 } features: { input_names: "device" feature_type: IdFeature embedding_dim: 16 hash_bucket_size: 500000 } features : { input_names: 'province' feature_type: IdFeature hash_bucket_size: 40 embedding_dim: 16 } features : { input_names: 'city' feature_type: IdFeature hash_bucket_size: 400 embedding_dim: 16 } features : { input_names: 'browser' feature_type: IdFeature hash_bucket_size: 40 embedding_dim: 16 } features : { input_names: 'network' feature_type: IdFeature hash_bucket_size: 4 embedding_dim: 16 } features : { input_names: 'os' feature_type: IdFeature hash_bucket_size: 40 embedding_dim: 16 } features : { input_names: 'version' feature_type: IdFeature hash_bucket_size: 20 embedding_dim: 16 } features : { input_names: 'item_id' feature_type: IdFeature hash_bucket_size: 10000 embedding_dim: 16 } features : { input_names: 'block' feature_type: IdFeature hash_bucket_size: 40 embedding_dim: 16 } features : { input_names: 'title' feature_type: SequenceFeature hash_bucket_size: 20000 embedding_dim: 16 sequence_combiner: { text_cnn:{ filter_sizes:[2,3,4] num_filters:[16,8,8] } }
} features : { input_names: 'query' feature_type: SequenceFeature hash_bucket_size: 200000 embedding_dim: 32 sequence_combiner: { text_cnn:{ filter_sizes:[2,3,4] num_filters:[16,8,8] } }
}
}
model_config:{ model_class: "DeepFM" feature_groups: { group_name: "deep" feature_names: "user_id" feature_names: "office_id" feature_names: "user_name" feature_names: "ip" feature_names: "device" feature_names: "province" feature_names: "city" feature_names: "browser" feature_names: "network" feature_names: "os" feature_names: "version" feature_names: "item_id" feature_names: "block" feature_names: "title" feature_names: "query" wide_deep:DEEP } feature_groups: { group_name: "wide" feature_names: "user_id" feature_names: "office_id" feature_names: "user_name" feature_names: "ip" feature_names: "device" feature_names: "province" feature_names: "city" feature_names: "browser" feature_names: "network" feature_names: "os" feature_names: "version" feature_names: "item_id" feature_names: "block" feature_names: "title" feature_names: "query" wide_deep:WIDE }
deepfm { wide_output_dim: 16
dnn {
hidden_units: [128, 64, 32]
}
final_dnn {
hidden_units: [128, 64,32]
}
l2_regularization: 1e-5
} embedding_regularization: 1e-7 } [2023-05-12 16:21:56,540][WARNING] pyhive is not installed. WARNING:tensorflow:From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/main.py:45: The name tf.GPUOptions is deprecated. Please use tf.compat.v1.GPUOptions instead.
[2023-05-12 16:21:57,039][WARNING] From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/main.py:45: The name tf.GPUOptions is deprecated. Please use tf.compat.v1.GPUOptions instead.
WARNING:tensorflow:From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/main.py:46: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
[2023-05-12 16:21:57,039][WARNING] From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/main.py:46: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
WARNING:tensorflow:From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/layers/layer_norm.py:9: The name tf.layers.Layer is deprecated. Please use tf.compat.v1.layers.Layer instead.
[2023-05-12 16:21:57,048][WARNING] From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/layers/layer_norm.py:9: The name tf.layers.Layer is deprecated. Please use tf.compat.v1.layers.Layer instead.
WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see: * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md * https://github.com/tensorflow/addons * https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.
[2023-05-12 16:21:57,051][WARNING] The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see: * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md * https://github.com/tensorflow/addons * https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.
WARNING:tensorflow:From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/input/csv_input.py:14: ignore_errors (from tensorflow.contrib.data.python.ops.error_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.data.experimental.ignore_errors(). [2023-05-12 16:21:57,364][WARNING] From /data/soft/anaconda3/envs/xurec/lib/python3.7/site-packages/easy_rec-0.6.2-py3.7.egg/easy_rec/python/input/csv_input.py:14: ignore_errors (from tensorflow.contrib.data.python.ops.error_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.data.experimental.ignore_errors(). easy_rec version: 0.6.2 Usage: easy_rec.help() WARNING:t
写回答
-
值得去的地方都没有捷径1.这个错误信息显示 KeyError: ‘title’，说明在特征组件中没有找到名为 ‘title’ 的特征列。您需要检查一下您的特征工程代码，确认是否正确地将 ‘title’ 这个特征列添加到了特征组件中。如果确认特征工程代码没有问题，您还需要检查一下您的数据是否包含 ‘title’ 这个特征列。如果数据中没有 ‘title’ 这个特征列，您需要先对数据进行预处理，将 ‘title’ 这个特征从原始数据中提取出来，并将其添加到特征工程中。 2.这段配置文件中没有明显的错误，但需要注意以下几点：
特征列的类型和名称是否正确，特别是在 feature_config 中，需要确保每个特征列的名称和类型与数据中的列名和类型一致。
DeepFM 模型中的参数设置是否合理，例如 embedding_dim、hidden_units、l2_regularization 等参数是否适合当前的数据集和模型结构。
训练和评估数据的路径是否正确，需要确保 train_input_path 和 eval_input_path 的路径与实际存储的数据路径一致。
物理网络蓝图和部署环境蓝图是否正确，需要确保环境变量和依赖库的配置正确。
您可以先检查以上几点，如果还有问题，请提供完整的错误信息，我将会尽力帮助您解决问题。
2023-05-15 07:54:27赞同 展开评论 打赏 -
针对问题1的回答：logview 和 config文件发一下看看。 针对问题2的回答：目前wide侧不支持添加SequenceFeature，所以需要把 title、query 从wide侧移除掉。此回答整理自钉群：“【EasyRec】推荐算法交流群”
2023-05-14 21:01:27赞同 展开评论 打赏
版权声明：本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
人工智能平台 PAI（Platform for AI，原机器学习平台PAI）是面向开发者和企业的机器学习/深度学习工程平台，提供包含数据标注、模型构建、模型训练、模型部署、推理优化在内的AI开发全链路服务，内置140+种优化算法，具备丰富的行业场景插件，为用户提供低门槛、高性能的云原生AI工程化能力。
