Update utils.py and Create test_utils_predict.py #566

2wins · 2018-05-13T12:22:05Z

Checklist

I've tested that my changes are compatible with the latest version of Tensorflow.
I've read the Contribution Guidelines
I've updated the documentation if necessary.

Motivation and Context

Fix #565
In #288, the error has not been handled properly.

Description

np.hstack makes data stacked along axis=1 while np.vstack along axis=0.

Fix tensorlayer#565

zsdonghao · 2018-05-13T12:39:54Z

@2wins thanks, I have one question. If the number of example can not be divided by the batch size, it is still work fine?

2wins · 2018-05-13T12:43:23Z

@zsdonghao Partially yes. Please check my comment in #288.

DEKHTIARJonathan · 2018-05-13T13:08:05Z

@2wins @zsdonghao why not trying to write a small unittests for all the cases you could think of?
If it is something tricky it would definitely be a good thing to have it tested.

BTW. In case you need smthg more versatile, np.stack is more generic and work just fine.

8000

zsdonghao · 2018-05-13T13:20:10Z

@DEKHTIARJonathan good idea. @2wins could you help to put the test in this PR?

Simply extend this one https://github.com/tensorlayer/tensorlayer/blob/master/tests/test_mnist_simple.py

2wins · 2018-05-13T13:23:10Z

@DEKHTIARJonathan @zsdonghao Please see the below snippets.

# case 1: No. of examples is not divisible by batch_size
# but the input placeholder's first dim is None.
x = tf.placeholder(tf.float32, [None, 5, 5, 3])
X = np.ones([127, 5, 5, 3])
net = tl.layers.InputLayer(x)
y = net.outputs
y_op = tf.nn.softmax(y)
result = tl.utils.predict(sess, net, X, x, y_op, batch_size=8)

# case 2: No. of examples > batch_size & not divisible by batch_size
x = tf.placeholder(tf.float32, [8, 5, 5, 3])
X = np.ones([127, 5, 5, 3])
net = tl.layers.InputLayer(x)
y = net.outputs
y_op = tf.nn.softmax(y)
result = tl.utils.predict(sess, net, X, x, y_op, batch_size=8)

# case 3: No. of examples < batch_size (actually same with the last mini-batch in case 2)
x = tf.placeholder(tf.float32, [8, 5, 5, 3])
X = np.ones([7, 5, 5, 3])
net = tl.layers.InputLayer(x)
y = net.outputs
y_op = tf.nn.softmax(y)
result = tl.utils.predict(sess, net, X, x, y_op, batch_size=8)

zsdonghao · 2018-05-13T13:27:37Z

@2wins nice! it would be great to add this test scripts into test folder ~

related with tensorlayer#288, tensorlayer#565, tensorlayer#566

DEKHTIARJonathan · 2018-05-13T17:18:38Z

@2wins please add the test you have created in #567 in this PR.
We try to only merge working PRs.

Please also update the file Changelog.md with your modifications. From now on, we will enforce this policy.

2wins · 2018-05-13T23:42:11Z

@DEKHTIARJonathan I see. Sorry to bother you.
@zsdonghao BTW, how can I join the Slack? Every invitation has not been sent.

DEKHTIARJonathan · 2018-05-14T06:59:27Z

There is a huge button in the readme. There is no way u can miss it.

DEKHTIARJonathan · 2018-05-14T07:01:37Z

BTW. As you can see the test is still failing. So there is an error either in your modifications or your test 😉.
That's exactly why I asked you to implement a test 😊

2wins · 2018-05-14T07:15:49Z

@DEKHTIARJonathan OK, I'll check it.
For Slack, can you invite me directly? The invitation mail is not sent to me.

related to tensorlayer#566

…atch-5

DEKHTIARJonathan · 2018-05-14T07:49:55Z

@2wins

DEKHTIARJonathan

Why most of the test is commented out ?
The modification is supposed to cover all cases right ?

DEKHTIARJonathan · 2018-05-14T07:52:29Z

CHANGELOG.md

-  - `tutorial_tfslim` added by (@2wins).
+  - `tutorial_tfslim` added (by @2wins).
+- Tests:
+  - `test_utils_predict.py` added (by @2wins).


Please mention why the test has been added: to insure that issue #??? is solved.

Actually, I didn't know what comment is proper and left a comment simply.
It is for the issue #288.

You can do the following:

- `test_utils_predict.py` added to reproduce and fix issue #288 (by @2wins).

The objective is just to have an idea about why you added this file.

2wins · 2018-05-14T07:52:32Z

@DEKHTIARJonathan case 2 and case 3 make an error. So I commented them out.

DEKHTIARJonathan · 2018-05-15T10:02:15Z

@2wins I believe that everyone can make mistakes beginners like experts. Maybe we can raise an explicit Exception "Batch_Size Mismatched bla bla bla" to make it easier to debug.

However, in my opinion, we should not in any case torture the library to make it user-error proof.

2wins · 2018-05-15T10:18:24Z

@DEKHTIARJonathan You're right. I completely have the same opinion about that. I think the explicit exception is the best solution here. Thanks.

DEKHTIARJonathan · 2018-05-15T10:22:38Z

You can use "AssertRaises" in the unittest to check that some case are raising an Exception ;)
Maybe raising a ValueError or something similar is the best thing to do.

2wins · 2018-05-15T13:57:52Z

@DEKHTIARJonathan Thanks. I'll apply it.

DEKHTIARJonathan · 2018-05-15T14:04:39Z

@2wins Please resolve the conflicts by copy-pasting the following, I can not do it on your fork.

# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

<!--

============== Guiding Principles ==============

* Changelogs are for humans, not machines.
* There should be an entry for every single version.
* The same types of changes should be grouped.
* Versions and sections should be linkable.
* The latest version comes first.
* The release date of each version is displayed.
* Mention whether you follow Semantic Versioning.

============== Types of changes (keep the order) ==============

* `Added` for new features.
* `Changed` for changes in existing functionality.
* `Deprecated` for soon-to-be removed features.
* `Removed` for now removed features.
* `Fixed` for any bug fixes.
* `Security` in case of vulnerabilities.
* `Dependencies Update` in case of vulnerabilities.
* `Contributors` to thank the contributors that worked on this PR.

============== How To Update The Changelog for a New Release ==============

** Always Keep The Unreleased On Top **

To release a new version, please update the changelog as followed:
1. Rename the `Unreleased` Section to the Section Number
2. Recreate an `Unreleased` Section on top
3. Update the links at the very bottom

======================= START: TEMPLATE TO KEEP IN CASE OF NEED ===================

** DO NOT MODIFY THIS SECTION ! **

## [Unreleased]

### Added

### Changed

### Deprecated

### Removed

### Fixed

### Security

### Dependencies Update

### Contributors

** DO NOT MODIFY THIS SECTION ! **

======================= END: TEMPLATE TO KEEP IN CASE OF NEED ===================

-->

<!-- YOU CAN EDIT FROM HERE -->

## [Unreleased]

### Added
- Tutorials:
  - `tutorial_tfslim` has been introduced to show how to use `SlimNetsLayer` (by @2wins in #560).
- Test:
  - `Layer_DeformableConvolution_Test` added to reproduce issue #572 with deformable convolution (by @DEKHTIARJonathan in #573)  
  - `test_utils_predict.py` added to reproduce and fix issue #288 (by @2wins in #566)
- CI Tool:
  - Danger CI has been added to enforce the update of the changelog (by @lgarithm and @DEKHTIARJonathan in #563)
  - https://github.com/apps/stale/ added to clean stale issues (by @DEKHTIARJonathan in #573)

### Changed
- Tensorflow CPU & GPU dependencies moved to separated requirement files in order to allow PyUP.io to parse them (by @DEKHTIARJonathan in #573)

### Deprecated

### Removed

### Fixed
- Issue #498 - Deprecation Warning Fix in `tl.layers.RNNLayer` with `inspect` (by @DEKHTIARJonathan in #574)
- Issue #498 - Deprecation Warning Fix in `tl.files` with truth value of an empty array is ambiguous (by @DEKHTIARJonathan in #575)
- Issue #572 with deformable convolution fixed (by @DEKHTIARJonathan in #573)
- Issue #565 related to `tl.utils.predict` fixed - `np.hstack` problem in which the results for multiple batches are stacked along `dim=1` (by @2wins in #566)

### Security

### Dependencies Update

### Contributors
@lgarithm @DEKHTIARJonathan @2wins


## [1.8.5] - 2018-05-09

### Added
- Github Templates added (by @DEKHTIARJonathan)
  - New issues Template
  - New PR Template
- Travis Deploy Automation on new Tag (by @DEKHTIARJonathan)
  - Deploy to PyPI and create a new version.
  - Deploy to Github Releases and upload the wheel files
- PyUP.io has been added to ensure we are compatible with the latest libraries (by @DEKHTIARJonathan)
- `deconv2d` now handling dilation_rate (by @zsdonghao)
- Documentation unittest added (by @DEKHTIARJonathan)
- `test_layers_core` has been added to ensure that `LayersConfig` is abstract.

### Changed
- All Tests Refactored - Now using unittests and runned with PyTest (by @DEKHTIARJonathan)
- Documentation updated (by @zsdonghao)
- Package Setup Refactored (by @DEKHTIARJonathan)
- Dataset Downlaod now using library progressbar2 (by @DEKHTIARJonathan)
- `deconv2d` function transformed into Class (by @zsdonghao)
- `conv1d` function transformed into Class (by @zsdonghao)
- super resolution functions transformed into Class (by @zsdonghao)
- YAPF coding style improved and enforced (by @DEKHTIARJonathan)

### Fixed
- Backward Compatibility Restored with deprecation warnings (by @DEKHTIARJonathan)
- Tensorflow Deprecation Fix (Issue #498):
  - AverageEmbeddingInputlayer (by @zsdonghao)
  - load_mpii_pose_dataset (by @zsdonghao)
- maxPool2D initializer issue #551 (by @zsdonghao)
- `LayersConfig` class has been enforced as abstract
- Pooling Layer Issue #557 fixed (by @zsdonghao)

### Dependencies Update
- scipy>=1.0,<1.1 => scipy>=1.1,<1.2

### Contributors
@zsdonghao @luomai @DEKHTIARJonathan

[Unreleased]: https://github.com/tensorlayer/tensorlayer/compare/1.8.5...master
[1.8.5]: https://github.com/tensorlayer/tensorlayer/compare/1.8.4...1.8.5

Not relevant anymore

DEKHTIARJonathan · 2018-05-15T14:10:44Z

@2wins you have to use the resolve conflicts function

Please copy+paste exactly the code I gave you by replacing everything ;)

2wins · 2018-05-15T14:12:46Z

@DEKHTIARJonathan I tried it first but Mark as resolved button was inactive even if there was no conflict. Now I've done it. Many Thanks.

DEKHTIARJonathan · 2018-05-15T14:14:13Z

CHANGELOG.md

@@ -72,6 +72,8 @@ To release a new version, please update the changelog as followed:
 - Tutorials:
  - `tutorial_tfslim` has been introduced to show how to use `SlimNetsLayer` (by @2wins in #560).
 - Test:
+  - `Layer_DeformableConvolution_Test` added to reproduce issue #572 with deformable convolution (by @DEKHTIARJonathan in #573)  


This line is a duplicate, please remove it.
I don't know how this one appeared here anyhow ^^

DEKHTIARJonathan · 2018-05-15T14:20:49Z

@zsdonghao @lgarithm can you please take 2 minutes to have a look.
I don't really why we had to change from np.hstack, so please verify that everything is correct ;)

To me the general structure of the PR is okay, I am unsure about np.concatenate vs np.hstack vs np.vstack. I let you have a look.

If it is good, you can remove [WIP] in the PR title, and you will be able to merge the PR

lgarithm · 2018-05-15T14:44:22Z

I don't use numpy much, but it looks OK from the document.
And it actually fixed the referred issue in comment.

LGTM

(Not that the layout of the result variable is changed.)

@zsdonghao I'll leave it to you to merge.

2wins · 2018-05-15T14:54:19Z

@DEKHTIARJonathan FYI,

np.concatenate: Join a sequence of arrays along an existing axis.
np.hstack: Stack arrays in sequence horizontally (column wise, axis=1)
np.vstack: Stack arrays in sequence vertically (row wise, axis=0)

np.hstack and np.vstack are implemented using np.concatenate.

2wins · 2018-05-17T15:09:39Z

@zsdonghao Please confirm this PR. Thanks.

* Update visualize.py * Update README.md Add an example for Adversarial Learning * Update more.rst Update the URLs * Update more.rst * Update example.rst Add the same link of BEGAN implementation. * Update example.rst * Update example.rst * Create tutorial_tfslim.py fixes #552 * Update tutorial_tfslim.py * Update utils.py Fix #565 * Update utils.py * Create test_utils_predict.py related with #288, #565, #566 * Create test_utils_predict.py * Update utils.py * Update test_utils_predict.py * Update CHANGELOG.md related to #566 * Update test_utils_predict.py * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py (fix Bad Coding Style) * Update test_utils_predict.py * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update convolution.py (Add AtrousConv2dTransLayer) * Add AtrousConv2dTransLayer * Fix some mistakes * Follow protocols * Fix coding style (yapf) * AtrousConv2dLayer fixed * AtrousConv2dTransposeLayer refactored * Fix coding style (yapf) * Fix error * Bias Add using premade tf func * Old TF Code Removed * Renamed to AtrousDeConv2dLayer * Update CHANGELOG.md * Release 1.8.6rc2 * Documentation Fix

* Update utils.py Fix #565 * Update utils.py * Create test_utils_predict.py * Update utils.py * Update test_utils_predict.py * Update CHANGELOG.md related to #566 * Update test_utils_predict.py * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py (fix Bad Coding Style) * Update test_utils_predict.py * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md

* Update visualize.py * Update README.md Add an example for Adversarial Learning * Update more.rst Update the URLs * Update more.rst * Update example.rst Add the same link of BEGAN implementation. * Update example.rst * Update example.rst * Create tutorial_tfslim.py fixes #552 * Update tutorial_tfslim.py * Update utils.py Fix #565 * Update utils.py * Create test_utils_predict.py related with #288, #565, #566 * Create test_utils_predict.py * Update utils.py * Update test_utils_predict.py * Update CHANGELOG.md related to #566 * Update test_utils_predict.py * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py * Update test_utils_predict.py (fix Bad Coding Style) * Update test_utils_predict.py * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update convolution.py (Add AtrousConv2dTransLayer) * Add AtrousConv2dTransLayer * Fix some mistakes * Follow protocols * Fix coding style (yapf) * AtrousConv2dLayer fixed * AtrousConv2dTransposeLayer refactored * Fix coding style (yapf) * Fix error * Bias Add using premade tf func * Old TF Code Removed * Renamed to AtrousDeConv2dLayer * Update CHANGELOG.md * Release 1.8.6rc2 * Documentation Fix

Update utils.py

10ce5d4

Fix tensorlayer#565

Update utils.py

151958e

2wins added a commit to 2wins/tensorlayer that referenced this pull request May 13, 2018

Create test_utils_predict.py

cef9310

related with tensorlayer#288, tensorlayer#565, tensorlayer#566

2wins mentioned this pull request May 13, 2018

Create test_utils_predict.py #567

Closed

3 tasks

2wins and others added 2 commits May 13, 2018 22:35

Merge branch 'master' into patch-5

445690e

Merge branch 'master' into patch-5

e504c85

Merge branch 'master' into patch-5

5d6ad68

zsdonghao previously approved these changes May 13, 2018

View reviewed changes

2wins changed the title ~~Update utils.py~~ Update utils.py and Create test_utils_predict.py May 14, 2018

Create test_utils_predict.py

ce3ef61

2wins added 2 commits May 14, 2018 16:14

Update utils.py

d274464

Update test_utils_predict.py

f494d8c

2wins added 3 commits May 14, 2018 16:23

Update CHANGELOG.md

619c08f

related to tensorlayer#566

Merge remote-tracking branch 'origin/patch-1' into patch-5

2546d64

Merge branch 'patch-5' of https://github.com/2wins/tensorlayer into p…

cb48f7d

…atch-5

DEKHTIARJonathan reviewed May 14, 2018

View reviewed changes

Update test_utils_predict.py

0cb228f

DEKHTIARJonathan changed the title ~~Update utils.py and Create test_utils_predict.py~~ [WIP] Update utils.py and Create test_utils_predict.py May 15, 2018

2wins added 2 commits May 15, 2018 23:08

Update CHANGELOG.md

cee22df

Update CHANGELOG.md

d3c4346

Merge branch 'master' into patch-5

cb1ebb7

DEKHTIARJonathan requested changes May 15, 2018

View reviewed changes

Update CHANGELOG.md

99fa99b

DEKHTIARJonathan approved these changes May 15, 2018

View reviewed changes

lgarithm approved these changes May 15, 2018

View reviewed changes

Update CHANGELOG.md

fd4a01c

DEKHTIARJonathan mentioned this pull request May 16, 2018

Reimplement OpenPose #585

Closed

Merge branch 'master' into patch-5

89e8535

zsdonghao changed the title ~~[WIP] Update utils.py and Create test_utils_predict.py~~ Update utils.py and Create test_utils_predict.py May 17, 2018

zsdonghao merged commit 67530b7 into tensorlayer:master May 17, 2018

DEKHTIARJonathan mentioned this pull request May 30, 2018

Release 1.8.6rc0 #660

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update utils.py and Create test_utils_predict.py #566

Update utils.py and Create test_utils_predict.py #566

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Update utils.py and Create test_utils_predict.py #566

Update utils.py and Create test_utils_predict.py #566

Uh oh!

Conversation

Uh oh!

Checklist

Motivation and Context

Description

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!