Document way to customize abbreviation (elipsis) on AssertionError #6682

ssbarnea · 2020-02-06T09:08:43Z

I keep getting errors that I cannot debug because the printed AssertionError is censored so much that makes impossible to see what was the difference.

Adding -vvvv did not help at all changing the way exceptions are rendered.

      assert l in out
E AssertionError: assert 'instance docker ansible default false false' in 'Instance Name Driver Name Provisioner Name Scenario Name Created Converged\n--------------- ---------...i-node false false\ninstance-2 docker ansible multi-node false false\n'

I did try to research the subject a lot online but I was not able to find any solution that would allow me to disable or customize when abbreviation happens, to make the limit upwards.

I raised this to PyTest because I am looking for a solution that can enable this at the entire test-suite and not on a specific file with tests, being able to do it globally is key for big projects.

Some resources I found:

https://stackoverflow.com/questions/50608443/how-do-i-get-pycharm-to-show-entire-error-diffs-from-pytest -- I was not able to find a way to hack DEFAULT_MAX_CHARS
https://stackoverflow.com/questions/38000993/how-can-i-get-my-assertions-in-pytest-to-stop-being-abbreviated-with-ellipsis

The text was updated successfully, but these errors were encountered:

blueyed · 2020-02-06T23:57:08Z

~~Try env CI=true …, but it does not help here.~~
(It is an annoying thing, I know - there are some issues/ideas in that regard open already)
Related: blueyed#63
The problem here is that the internal pytest_assertrepr_compare hook does nothing here (for `op = "in"), and the assertion rewrite having it truncated already anyway.
Somehow related docs: https://github.com/blueyed/pytest/blob/49f6aaf725610acf67087b0a38dc69e636c7402f/doc/en/assert.rst#defining-your-own-explanation-for-failed-assertions

ssbarnea · 2021-03-02T15:21:35Z

One year later and I am facing again the same issue, a bug caused by an incomplete feature, one that cannot be disabled, the truncation.

Can we please do something about it? For some projects this is testing PITA as the comparisons are almost always bigger than the limit and the user is left clueless to face the what is behind those ellipses? questions.

Fix pytest-dev#6682

nicoddemus · 2021-03-03T00:54:11Z

Indeed this is frustrating, thanks @ssbarnea for bringing it to light again.

I've opened #8391 as an attempt to fix this, feedback welcome.

ssbarnea · 2021-03-03T10:55:55Z

I think it was #8391 -- going to test it now.

nicoddemus · 2021-03-03T11:43:36Z

Oops yes, fixed my comment.

Fix pytest-dev#6682

Fix pytest-dev#6682 Fix pytest-dev#8403

ssbarnea · 2024-03-12T12:31:54Z

@nicoddemus I think that we need to reopen this issue because current behavior is still not ok. I found multiple similar unanswered question online such https://stackoverflow.com/questions/38000993/how-can-i-get-my-assertions-in-pytest-to-stop-being-abbreviated-with-ellipsis

IMHO, I think that we need to make these values configurable regardless of the verbosity level, especially as we know that changing general verbosity level has other side effects.

My opinion is that most people would want to run in minimal verbosity level (especially) on ci but have verbose errors for failed tests.

If I run in verbose by default displaying the console log in the browser can be problematic, true for github actions and also most CI/CD system I know (none behaves nice with very long console logs).

bluetech · 2024-03-12T12:36:50Z

@ssbarnea Check out https://docs.pytest.org/en/stable/reference/reference.html#confval-verbosity_assertions, added in pytest 8.

ssbarnea · 2024-05-09T16:22:01Z

@bluetech That issue seems to never want to be go away... :p --- I tried, still same output with ellipses, in fact even calling pytest -vvvv does ellipsis....

[tool.pytest.ini_options]
verbosity_assertions = 4
verbosity_test_cases = 4

Output:

>       assert len(results) == len(expected), results
E       AssertionError: [[no-tabs] (Most files should not contain tabs.) matched examples/playbooks/rule-no-tabs.yml:10 Task/Handler: Foo, [no...hould not contain tabs.) matched examples/playbooks/rule-no-tabs.yml:37 Task/Handler: Should not trigger no-tabs rules]
E       assert 3 == 2
E        +  where 3 = len([[no-tabs] (Most files should not contain tabs.) matched examples/playbooks/rule-no-tabs.yml:10 Task/Handler: Foo, [no-tabs] (Most files should not contain tabs.) matched examples/playbooks/rule-no-tabs.yml:13 Task/Handler: Key has a tab, [no-tabs] (Most files should not contain tabs.) matched examples/playbooks/rule-no-tabs.yml:37 Task/Handler: Should not trigger no-tabs rules])
E        +  and   2 = len((13, 'Most files should not contain tabs.'))

Using latest 8.2.0. I put some breakpoints and tried to find what happens. Apparently the DEFAULT_REPR_MAX_SIZE = 240 and saferepr() is causing it to happen anyway:

  /Users/ssbarnea/c/os/pytest/src/_pytest/python.py(162)pytest_pyfunc_call()
    161     testargs = {arg: funcargs[arg] for arg in pyfuncitem._fixtureinfo.argnames}
--> 162     result = testfunction(**testargs)
    163     if hasattr(result, "__await__") or hasattr(result, "__aiter__"):

  /Users/ssbarnea/c/a/ansible-lint/src/ansiblelint/rules/no_tabs.py(92)test_no_tabs_rule()
     90             assert results[i].lineno == expected[0]
     91             assert results[i].message == expected[1]
---> 92         assert len(results) == len(expected), results

  /Users/ssbarnea/c/os/pytest/src/_pytest/assertion/rewrite.py(454)_format_assertmsg()
    453     if not isinstance(obj, str):
--> 454         obj = saferepr(obj)
    455         replaces.append(("\\n", "\n~"))

  /Users/ssbarnea/c/os/pytest/src/_pytest/_io/saferepr.py(112)saferepr()
    111     """
--> 112     return SafeRepr(maxsize, use_ascii).repr(obj)
    113 

  /Users/ssbarnea/c/os/pytest/src/_pytest/_io/saferepr.py(62)repr()
     61             else:
---> 62                 s = super().repr(x)
     63 

  /Users/ssbarnea/.asdf/installs/python/3.12.2/lib/python3.12/reprlib.py(59)repr()
     58     def repr(self, x):
---> 59         return self.repr1(x, self.maxlevel)
     60 

  /Users/ssbarnea/.asdf/installs/python/3.12.2/lib/python3.12/reprlib.py(67)repr1()
     66         if hasattr(self, 'repr_' + typename):
---> 67             return getattr(self, 'repr_' + typename)(x, level)
     68         else:

  /Users/ssbarnea/.asdf/installs/python/3.12.2/lib/python3.12/reprlib.py(110)repr_list()
    109     def repr_list(self, x, level):
--> 110         return self._repr_iterable(x, level, '[', ']', self.maxlist)
    111 

  /Users/ssbarnea/.asdf/installs/python/3.12.2/lib/python3.12/reprlib.py(98)_repr_iterable()
     97             repr1 = self.repr1
---> 98             pieces = [repr1(elem, newlevel) for elem in islice(x, maxiter)]
     99             if n > maxiter:

  /Users/ssbarnea/.asdf/installs/python/3.12.2/lib/python3.12/reprlib.py(69)repr1()
     68         else:
---> 69             return self.repr_instance(x, level)
     70 

  /Users/ssbarnea/c/os/pytest/src/_pytest/_io/saferepr.py(80)repr_instance()
     79         if self.maxsize is not None:
---> 80             s = _ellipsize(s, self.maxsize)
     81         return s

> /Users/ssbarnea/c/os/pytest/src/_pytest/_io/saferepr.py(29)_ellipsize()
     28     breakpoint()
---> 29     if len(s) > maxsize:
     30         i = max(0, (maxsize - 3) // 2)

Fixes: #6682

Zac-HD added topic: reporting related to terminal output and user-facing messages and errors type: question general question, might be closed after 2 weeks of inactivity labels Feb 6, 2020

Zac-HD added topic: rewrite related to the assertion rewrite mechanism type: bug problem that needs to be addressed and removed type: question general question, might be closed after 2 weeks of inactivity labels Feb 22, 2020

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Mar 3, 2021

Increase truncation threshold with -vvv, disable with -vvvv

d81aeb5

Fix pytest-dev#6682

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Mar 3, 2021

Increase truncation threshold with -vvv, disable with -vvvv

5b87a59

Fix pytest-dev#6682

nicoddemus mentioned this issue Mar 3, 2021

Increase truncation threshold with -v, disable with -vv #8391

Merged

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Mar 3, 2021

Increase truncation threshold with -vvv, disable with -vvvv

3d4c3d8

Fix pytest-dev#6682

nicoddemus mentioned this issue Mar 5, 2021

Truncation threshold at different verbosity levels #8403

Closed

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Mar 20, 2021

Increase truncation threshold with -vvv, disable with -vvvv

baf9703

Fix pytest-dev#6682

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Mar 26, 2021

Increase truncation threshold with -v, disable with -vv

be8d63e

Fix pytest-dev#6682 Fix pytest-dev#8403

nicoddemus closed this as completed in #8391 Mar 26, 2021

ssbarnea mentioned this issue Mar 12, 2024

Make truncation threshold (ellipsis) configurable regardless the verbosity level #12111

Closed

ssbarnea reopened this May 10, 2024

ssbarnea added a commit that referenced this issue May 14, 2024

Allow override of DEFAULT_REPR_MAX_SIZE

a02ac43

Fixes: #6682

ssbarnea mentioned this issue May 14, 2024

WIP: Truncate objects the same way we truncate other messages #12322

Draft

ssbarnea added a commit that referenced this issue May 14, 2024

Allow override of DEFAULT_REPR_MAX_SIZE

65b3524

Fixes: #6682

The-Compiler mentioned this issue May 14, 2024

Extra info in assertion is truncated even with -vv #12307

Open

ssbarnea added a commit that referenced this issue May 14, 2024

Call correct _saferepr() in _format_assertmsg

66a18f7

Fixes: #6682

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document way to customize abbreviation (elipsis) on AssertionError #6682

Document way to customize abbreviation (elipsis) on AssertionError #6682

ssbarnea commented Feb 6, 2020

blueyed commented Feb 6, 2020 •

edited

ssbarnea commented Mar 2, 2021

nicoddemus commented Mar 3, 2021 •

edited

ssbarnea commented Mar 3, 2021 •

edited

nicoddemus commented Mar 3, 2021

ssbarnea commented Mar 12, 2024

bluetech commented Mar 12, 2024

ssbarnea commented May 9, 2024 •

edited

Document way to customize abbreviation (elipsis) on AssertionError #6682

Document way to customize abbreviation (elipsis) on AssertionError #6682

Comments

ssbarnea commented Feb 6, 2020

blueyed commented Feb 6, 2020 • edited

ssbarnea commented Mar 2, 2021

nicoddemus commented Mar 3, 2021 • edited

ssbarnea commented Mar 3, 2021 • edited

nicoddemus commented Mar 3, 2021

ssbarnea commented Mar 12, 2024

bluetech commented Mar 12, 2024

ssbarnea commented May 9, 2024 • edited

blueyed commented Feb 6, 2020 •

edited

nicoddemus commented Mar 3, 2021 •

edited

ssbarnea commented Mar 3, 2021 •

edited

ssbarnea commented May 9, 2024 •

edited