Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[onert] This draft adds unittests of nnapi_api for training #12985

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

ragmani
Copy link
Contributor

@ragmani ragmani commented May 13, 2024

ONE-DCO-1.0-Signed-off-by: ragmani ragmani0216@gmail.com

@ragmani
Copy link
Contributor Author

ragmani commented Jun 4, 2024

An error occurred on tizen build

[2024-06-04T04:22:04.931Z] [  827s] /home/abuild/rpmbuild/BUILD/nnfw-1.27.0/runtime/onert/backend/train/MemoryPlanner.test.cc:17:10: fatal error: gtest/gtest.h: No such file or directory
[2024-06-04T04:22:04.931Z] [  827s]    17 | #include <gtest/gtest.h>
[2024-06-04T04:22:04.931Z] [  827s]       |          ^~~~~~~~~~~~~~~
[2024-06-04T04:22:04.931Z] [  827s] compilation terminated.
[2024-06-04T04:22:04.931Z] [  827s] gmake[2]: *** [runtime/onert/backend/train/CMakeFiles/test_onert_backend_train.dir/build.make:76: runtime/onert/backend/train/CMakeFiles/test_onert_backend_train.dir/MemoryPlanner.test.cc.o] Error 1
[2024-06-04T04:22:04.931Z] [  827s] gmake[2]: *** Waiting for unfinished jobs....
[2024-06-04T04:22:04.931Z] [  827s] /home/abuild/rpmbuild/BUILD/nnfw-1.27.0/runtime/onert/backend/train/optimizer/Optimizers.test.cc:23:10: fatal error: gtest/gtest.h: No such file or directory
[2024-06-04T04:22:04.931Z] [  827s]    23 | #include <gtest/gtest.h>
[2024-06-04T04:22:04.931Z] [  827s]       |          ^~~~~~~~~~~~~~~
[2024-06-04T04:22:04.931Z] [  827s] compilation terminated.

@ragmani ragmani force-pushed the draft/add_training_unittests branch from 80269f8 to 3788bb9 Compare June 4, 2024 09:59
ragmani added 11 commits June 5, 2024 09:18
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
@ragmani ragmani force-pushed the draft/add_training_unittests branch from 6073fdd to 572e0ab Compare June 5, 2024 01:06
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
_context->addTrainCase(
uniformTCD<float>({{{1, 2, 7, 8, 3, 4, 9, 1, 5, 6, 11, 2}}}, // input dataset
{{{1, -4, 1, -3, 2, -2, 2, -4}}}, // expected dataset
{13.107666f} // last losses
Copy link
Contributor Author

@ragmani ragmani Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test on tensorflow with learning rate(0.01f)

Epoch 1/5
1/1 [==============================] - 0s 157ms/step - loss: 6.8750 - categorical_accuracy: 1.0000
Epoch 2/5
1/1 [==============================] - 0s 2ms/step - loss: 2.5275 - categorical_accuracy: 1.0000
Epoch 3/5
1/1 [==============================] - 0s 2ms/step - loss: 1.6320 - categorical_accuracy: 1.0000
Epoch 4/5
1/1 [==============================] - 0s 2ms/step - loss: 1.1701 - categorical_accuracy: 1.0000
Epoch 5/5
1/1 [==============================] - 0s 2ms/step - loss: 0.9224 - categorical_accuracy: 1.0000
==========================

This comment was marked as duplicate.

_context->addTrainCase(
uniformTCD<float>({{{0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f}}}, // input dataset
{{{6.5f, 7.5f, 8.5f, 3.5f, 8.5f, 5.5f, 2.5f, 3.5f}}}, // expected dataset
{74.799194f} // last losses
Copy link
Contributor Author

@ragmani ragmani Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
1/1 [==============================] - 0s 154ms/step - loss: 38.0000 - categorical_accuracy: 0.5000
Epoch 2/5
1/1 [==============================] - 0s 2ms/step - loss: 26.6868 - categorical_accuracy: 0.5000
Epoch 3/5
1/1 [==============================] - 0s 2ms/step - loss: 19.8101 - categorical_accuracy: 0.5000
Epoch 4/5
1/1 [==============================] - 0s 1ms/step - loss: 15.5431 - categorical_accuracy: 0.7500
Epoch 5/5
1/1 [==============================] - 0s 1ms/step - loss: 12.8424 - categorical_accuracy: 0.7500
==========================

This comment was marked as duplicate.

_context->addTrainCase(
uniformTCD<float>({{{0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f}}}, // input dataset
{{{6.0f, 6.0f, 6.0f, 6.0f, 6.0f, 6.0f, 6.0f, 6.0f}}}, // expected dataset
{72.f} // last losses
Copy link
Contributor Author

@ragmani ragmani Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
1/1 [==============================] - 0s 158ms/step - loss: 36.0000 - categorical_accuracy: 1.0000
Epoch 2/5
1/1 [==============================] - 0s 2ms/step - loss: 36.0000 - categorical_accuracy: 1.0000
Epoch 3/5
1/1 [==============================] - 0s 2ms/step - loss: 36.0000 - categorical_accuracy: 1.0000
Epoch 4/5
1/1 [==============================] - 0s 2ms/step - loss: 36.0000 - categorical_accuracy: 1.0000
Epoch 5/5
1/1 [==============================] - 0s 2ms/step - loss: 36.0000 - categorical_accuracy: 1.0000
==========================

This comment was marked as duplicate.

_context->addTrainCase(uniformTCD<float>(
{{{0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f}}}, // input dataset
{{{6.0f, 16.0f, 8.0f, 16.0f, 10.0f, 16.0f, 12.0f, 16.0f}}}, // expected dataset
{329.703643f} // last losses
Copy link
Contributor Author

@ragmani ragmani Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
1/1 [==============================] - 0s 154ms/step - loss: 171.0000 - categorical_accuracy: 0.0000e+00
Epoch 2/5
1/1 [==============================] - 0s 2ms/step - loss: 69.5150 - categorical_accuracy: 1.0000
Epoch 3/5
1/1 [==============================] - 0s 2ms/step - loss: 29.9159 - categorical_accuracy: 1.0000
Epoch 4/5
1/1 [==============================] - 0s 2ms/step - loss: 13.7338 - categorical_accuracy: 1.0000
Epoch 5/5
1/1 [==============================] - 0s 2ms/step - loss: 6.7380 - categorical_accuracy: 1.0000
==========================

This comment was marked as duplicate.

_context->addTrainCase(
uniformTCD<float>({{{1, 3}}, {{2, 1}}}, // inputs
{{{2, 1, 5, 5, 2, 1, 5, 5}}, {{2, 1, 5, 5, 2, 1, 5, 6}}}, // expected
{21.520715f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 3ms/step - loss: 14.2234 - categorical_accuracy: 0.0000e+00
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 13.2278 - categorical_accuracy: 0.5000
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 12.3045 - categorical_accuracy: 0.5000
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 11.4484 - categorical_accuracy: 0.5000
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 10.6544 - categorical_accuracy: 0.5000
==========================

_context->addTrainCase(
uniformTCD<float>({{{1, 3, 2, 1}}}, // inputs
{{{2, 1, 5, 5, 2, 1, 5, 5, 2, 1, 5, 5, 2, 1, 5, 6}}}, // expected
{{43.178924f}} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
1/1 [==============================] - 0s 172ms/step - loss: 14.4375 - categorical_accuracy: 0.0000e+00
Epoch 2/5
1/1 [==============================] - 0s 1ms/step - loss: 13.9950 - categorical_accuracy: 0.5000
Epoch 3/5
1/1 [==============================] - 0s 1ms/step - loss: 13.5668 - categorical_accuracy: 0.5000
Epoch 4/5
1/1 [==============================] - 0s 1ms/step - loss: 13.1523 - categorical_accuracy: 0.5000
Epoch 5/5
1/1 [==============================] - 0s 1ms/step - loss: 12.7512 - categorical_accuracy: 0.5000
==========================

ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
@ragmani ragmani force-pushed the draft/add_training_unittests branch from 9b37461 to 6c64128 Compare June 12, 2024 07:03
_context->addTrainCase(uniformTCD<float>(
{{{1, 3}, {0, 1, 2, 3, 4, 5, 6, 7}}, {{2, 1}, {7, 6, 5, 4, 3, 2, 1, 0}}}, // inputs
{{{2, 1, 5, 5, 2, 1, 5, 5}}, {{2, 1, 5, 5, 2, 1, 5, 6}}}, // expected
{8.4678f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 3ms/step - loss: 9.2218 - categorical_accuracy: 0.0000e+00
Epoch 2/5
2/2 [==============================] - 0s 2ms/step - loss: 8.9554 - categorical_accuracy: 0.0000e+00
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 8.7044 - categorical_accuracy: 0.0000e+00
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 8.4678 - categorical_accuracy: 0.0000e+00
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 8.2444 - categorical_accuracy: 0.0000e+00
==========================

_context->addTrainCase(uniformTCD<float>(
{{{0, 1, 2, 3, 4, 5, 1, 3}, {6, 7}}, {{5, 4, 3, 2, 1, 0, 2, 1}, {7, 6}}}, // inputs
{{{2, 1, 5, 5, 2, 1, 5, 5}}, {{2, 1, 5, 5, 2, 1, 5, 6}}}, // expected
{3.2863f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 3ms/step - loss: 7.3265 - categorical_accuracy: 0.0000e+00
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 4.6811 - categorical_accuracy: 0.0000e+00
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 3.6735 - categorical_accuracy: 0.0000e+00
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 3.2863 - categorical_accuracy: 0.0000e+00
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 3.1329 - categorical_accuracy: 0.0000e+00
==========================

_context->addTrainCase(
uniformTCD<float>({{{0, 3}, {6, 7}}, {{5, 4}, {7, 6}}}, // inputs
{{{3, 2, 1, 2, 5, 6, 1, 0}}, {{2, 1, 5, 5, 2, 1, 5, 6}}}, // expected
{12.2822f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 3ms/step - loss: 12.5488 - categorical_accuracy: 0.0000e+00
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 12.4590 - categorical_accuracy: 0.0000e+00
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 12.3701 - categorical_accuracy: 0.0000e+00
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 12.2822 - categorical_accuracy: 0.0000e+00
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 12.1951 - categorical_accuracy: 0.0000e+00
==========================

ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
@ragmani ragmani force-pushed the draft/add_training_unittests branch from 81e7c83 to 813c99c Compare June 13, 2024 09:39
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
ONE-DCO-1.0-Signed-off-by: ragmani <ragmani0216@gmail.com>
@ragmani ragmani force-pushed the draft/add_training_unittests branch from fa89c6a to 53ccdcd Compare June 14, 2024 06:18
_context = std::make_unique<GenModelTrainContext>(cgen.finish());
_context->addTrainCase(uniformTCD<float>({{{1, 3}}, {{2, 1}}}, // inputs
{{{5}}, {{3}}}, // expected
{13.3691f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 4ms/step - loss: 16.7778 - categorical_accuracy: 1.0000
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 15.5544 - categorical_accuracy: 1.0000
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 14.4203 - categorical_accuracy: 1.0000
Epoch 4/5
2/2 [==============================] - 0s 2ms/step - loss: 13.3691 - categorical_accuracy: 1.0000
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 12.3947 - categorical_accuracy: 1.0000

_context->addTrainCase(
uniformTCD<float>({{{0, 1, 2, 3, 4, 5, 6, 7}}, {{7, 6, 5, 4, 3, 2, 1, 0}}}, // inputs
{{{0, 13, 52, 0}}, {{0, 31, 24, 0}}}, // expected
{1.3899f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 3ms/step - loss: 462.7862 - categorical_accuracy: 0.0000e+00
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 32.8115 - categorical_accuracy: 0.5000
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 5.8401 - categorical_accuracy: 1.0000
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 1.3900 - categorical_accuracy: 1.0000
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 0.3422 - categorical_accuracy: 1.0000

_context->addTrainCase(
uniformTCD<float>({{{1, 3}}, {{2, 1}}}, // inputs
{{{0, 1, 0, 0, 0, 0, 0, 0}}, {{0, 0, 0, 0, 0, 1, 0, 0}}}, // expected
{0.1092f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 3ms/step - loss: 0.1094 - categorical_accuracy: 0.0000e+00
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 0.1093 - categorical_accuracy: 0.5000
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 0.1092 - categorical_accuracy: 0.5000
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 0.1092 - categorical_accuracy: 0.5000
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 0.1091 - categorical_accuracy: 0.5000

_context->addTrainCase(
uniformTCD<float>({{{1, 3}}, {{2, 1}}}, // inputs
{{{0, 1, 5, 5, 2, 1, 5, 5}}, {{2, 1, 5, 5, 0, 1, 5, 6}}}, // expected
{13.5010f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 4ms/step - loss: 13.9041 - categorical_accuracy: 0.0000e+00
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 13.7684 - categorical_accuracy: 0.5000
Epoch 3/5
2/2 [==============================] - 0s 2ms/step - loss: 13.6340 - categorical_accuracy: 0.5000
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 13.5010 - categorical_accuracy: 0.5000
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 13.3693 - categorical_accuracy: 0.5000

_context->addTrainCase(
uniformTCD<float>({{{1, 3}}, {{2, 1}}}, // inputs
{{{2, 0, 6, 5, 2, 1, 6, 5}}, {{2, 1, 6, 5, 0, 1, 6, 6}}}, // expected
{16.3412f} // loss
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
2/2 [==============================] - 0s 3ms/step - loss: 16.8338 - categorical_accuracy: 0.5000
Epoch 2/5
2/2 [==============================] - 0s 1ms/step - loss: 16.6680 - categorical_accuracy: 1.0000
Epoch 3/5
2/2 [==============================] - 0s 1ms/step - loss: 16.5038 - categorical_accuracy: 1.0000
Epoch 4/5
2/2 [==============================] - 0s 1ms/step - loss: 16.3412 - categorical_accuracy: 1.0000
Epoch 5/5
2/2 [==============================] - 0s 1ms/step - loss: 16.1803 - categorical_accuracy: 1.0000

1, -2, 2, 4, -4, 2, 2, 0, 4, -1, -2, 4}}}, // input dataset
{{{47, -4, -25, 9, 10, 10, -13, 11, -14, -26, -12, 26, 20, 40, 1, 3, 11,
4}}}, // expected dataset
{226.5260f} // last losses
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epoch 1/5
1/1 [==============================] - 0s 163ms/step - loss: 403.3333 - categorical_accuracy: 1.0000
Epoch 2/5
1/1 [==============================] - 0s 2ms/step - loss: 324.0978 - categorical_accuracy: 0.0000e+00 
Epoch 3/5 
1/1 [==============================] - 0s 2ms/step - loss: 267.7882 - categorical_accuracy: 0.0000e+00 
Epoch 4/5 
1/1 [==============================] - 0s 2ms/step - loss: 226.5260 - categorical_accuracy: 0.0000e+00 
Epoch 5/5 
1/1 [==============================] - 0s 2ms/step - loss: 195.3313 - categorical_accuracy: 0.0000e+00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants