Discussion:
bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.
(too old to reply)
Bob
2010-02-12 22:52:31 UTC
Permalink
As I work with text in C++ mode the indentation begins to deteriorate. For
example, TAB on the class line wrongly indents it:



template<class _Enum>

class Build<TL::End,_Enum>



I work-around the problem by writing the buffer, killing the buffer, then
visiting the file and re-indenting with TAB on the class line to get the
expected indentation:



template<class _Enum>

class Build<TL::End,_Enum>



Same thing happens with ESC C-q.



My impression is there is internal state which doesn't get cleared as I work
the text toward a syntax correct unit, causing incorrect indentation. The
bad state is ultimately cleared by re-visiting the file.



Sorry I can't come up with something reproducible.



Has anyone else seen this issue?
Chong Yidong
2010-02-13 07:37:42 UTC
Permalink
Hi Alan,

Looks like this is related to Bug#5490. WDYT?
Post by Bob
As I work with text in C++ mode the indentation begins to deteriorate. For
template<class _Enum>
class Build<TL::End,_Enum>
I work-around the problem by writing the buffer, killing the buffer, then
visiting the file and re-indenting with TAB on the class line to get the
template<class _Enum>
class Build<TL::End,_Enum>
Same thing happens with ESC C-q.
My impression is there is internal state which doesn't get cleared as I work
the text toward a syntax correct unit, causing incorrect indentation. The bad
state is ultimately cleared by re-visiting the file.
Sorry I can't come up with something reproducible.
Has anyone else seen this issue?
Chong Yidong
2010-02-14 05:09:39 UTC
Permalink
Forwarding an email with pertinent info to the bug tracker, and to Alan
(i) Have you ever noticed this bug happening on earlier Emacsen, in
particular Emacs 23.1?
Yes, it happened in 23.1 also.
(ii) Does this bug happen every time you use Emacs this long, or only
sometimes?
For me it happens within minutes of use. My C++ files are HEAVILY templated
and the issue appears to be related to '<' and '>' of template args, e.g.
typedef IF<
MemberOf<a,b>::value,
IF<
Length<c,d>::value,
true,
false
::Result,
false
::Result Result;
I will have several of these, one after the other, and be editing
them. By accident I'll leave out a ',' or a '>' or kill & yank lines
with similar error. After minutes of such editing, the indentation
gets wacked. Sometimes by TAB indentation on a prior line the problem
corrects. Sometimes I have to write the buffer and re-visit it.
(iv) Are all C(++) buffers in this state, or only some? Any correlation
with how many edits have been done on a buffer, or how long it's been
loaded?
Just the buffer with the '<' and '>' in it.
Alan Mackenzie
2010-02-14 20:10:06 UTC
Permalink
Hi, Bob and Yidong,
Post by Chong Yidong
Forwarding an email with pertinent info to the bug tracker, and to Alan
I have the horrible suspicion that this is different from 5490. :-(
Post by Chong Yidong
(i) Have you ever noticed this bug happening on earlier Emacsen, in
particular Emacs 23.1?
Yes, it happened in 23.1 also.
(ii) Does this bug happen every time you use Emacs this long, or
only sometimes?
For me it happens within minutes of use. My C++ files are HEAVILY
templated and the issue appears to be related to '<' and '>' of
template args, e.g.
typedef IF<
MemberOf<a,b>::value,
IF<
Length<c,d>::value,
true,
false
::Result,
false
::Result Result;
OK, template brackets are difficult, because they're so difficult to
distinguish from less/greater than.

Bob, seeing as how the bug hits you "withing minutes", is there any
chance you could try and get it reproducible? Using a dribble file might
be useful (see the page "Checklist" in the Emacs manual; you'd use this
with something like '(open-dribble-file "~/dribble")' at the beginning of
your Emacs session).

If you manage to get it reproducible, could you send me the C++ file
(which I will regard as confidential if needed) together with the
sequence of commands (e.g. the dribble file) which triggers the bug.
Also appreciated would be a dump of your CC Mode configuration, generated
by C-c C-b.
Post by Chong Yidong
I will have several of these, one after the other, and be editing
them. By accident I'll leave out a ',' or a '>' or kill & yank lines
with similar error. After minutes of such editing, the indentation
gets wacked. Sometimes by TAB indentation on a prior line the problem
corrects. Sometimes I have to write the buffer and re-visit it.
One other thing. Would you please try M-: (c-state-cache-init) when in
this state, and tell me if it resolves the badness. Thanks!
Post by Chong Yidong
(iv) Are all C(++) buffers in this state, or only some? Any correlation
with how many edits have been done on a buffer, or how long it's been
loaded?
Just the buffer with the '<' and '>' in it.
--
Alan Mackenzie (Nuremberg, Germany).
Bob
2010-02-15 22:15:21 UTC
Permalink
More stuff:

With the emacs I've had going since last week I have several buffers where
the
indentation is incorrect. I validate by writing the buffer, then in a fresh
emacs,
visit the file, go to the lines with incorrect indentation, apply TAB and
the
result is the correct indentation. Ok, nothing new there.

This is new: After applying TAB in the 2nd emacs to correct the indentation
I write the file, then go back to the original buffer and re-visit the file
with "C-x C-f" "ESC-n".
Emacs confirms the file has changed, I reply "yes" and the visible buffer
becomes truncated
at the line where I TAB'ed to reformat:

The saved file is:

template<
class _TypeKind,
class _EStore = typename ScalarVectorSelector<_TypeKind>::Result
bool _fixed = TL::MemberOf<
DataTypeTrait<GetDataType<_TypeKind>::Result>::TAbout,
EnumDTP::FixedPacked
::value
The re-visited buffer now shows:

template<
class _TypeKind,
class _EStore = typename ScalarVectorSelector<_TypeKind>::Result
bool _fixed = TL::MemberOf<

EOF <--------------- NO MORE TEXT!

The original incorrect indentation was something like this (it got lost when
I revisited):

template<
class _TypeKind,
class _EStore = typename ScalarVectorSelector<_TypeKind>::Result
bool _fixed = TL::MemberOf<
DataTypeTrait<GetDataType<_TypeKind>::Result>::TAbout,
EnumDTP::FixedPacked
::value
Now, I kill the truncated buffer, re-visit (the original emacs) and I see
correct indentation of the saved file, so I can continue work :) in the
original emacs.


-----Original Message-----
From: Alan Mackenzie [mailto:***@muc.de]
Sent: Sunday, February 14, 2010 12:10 PM
To: Chong Yidong
Cc: Bob; ***@debbugs.gnu.org
Subject: Re: bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting
issue.

Hi, Bob and Yidong,
Forwarding an email with pertinent info to the bug tracker, and to Alan
I have the horrible suspicion that this is different from 5490. :-(
(i) Have you ever noticed this bug happening on earlier Emacsen, in
particular Emacs 23.1?
Yes, it happened in 23.1 also.
(ii) Does this bug happen every time you use Emacs this long, or
only sometimes?
For me it happens within minutes of use. My C++ files are HEAVILY
templated and the issue appears to be related to '<' and '>' of
template args, e.g.
typedef IF<
MemberOf<a,b>::value,
IF<
Length<c,d>::value,
true,
false
::Result,
false
::Result Result;
OK, template brackets are difficult, because they're so difficult to
distinguish from less/greater than.

Bob, seeing as how the bug hits you "withing minutes", is there any
chance you could try and get it reproducible? Using a dribble file might
be useful (see the page "Checklist" in the Emacs manual; you'd use this
with something like '(open-dribble-file "~/dribble")' at the beginning of
your Emacs session).

If you manage to get it reproducible, could you send me the C++ file
(which I will regard as confidential if needed) together with the
sequence of commands (e.g. the dribble file) which triggers the bug.
Also appreciated would be a dump of your CC Mode configuration, generated
by C-c C-b.
I will have several of these, one after the other, and be editing
them. By accident I'll leave out a ',' or a '>' or kill & yank lines
with similar error. After minutes of such editing, the indentation
gets wacked. Sometimes by TAB indentation on a prior line the problem
corrects. Sometimes I have to write the buffer and re-visit it.
One other thing. Would you please try M-: (c-state-cache-init) when in
this state, and tell me if it resolves the badness. Thanks!
(iv) Are all C(++) buffers in this state, or only some? Any correlation
with how many edits have been done on a buffer, or how long it's been
loaded?
Just the buffer with the '<' and '>' in it.
--
Alan Mackenzie (Nuremberg, Germany).
Alan Mackenzie
2010-02-16 16:39:24 UTC
Permalink
Hi, Bob,
Alan,
I am now running two emacs 23.1.91.1, one I started last week with many
files open in which I "work" etc., and one I just started with dribbling
"on" to edit a file I have seen the issue in. After half-hours editing in
this one, nothing bad has happened. I temporarily jumped back to the "work"
session to add a few needed lines and wammy! the problem appeared after
Could you be a little bit more exact in exactly what you did here. I
have just fired up emacs 23.1.91, typed in

namespace foo
{

(in column 0), then typed in the the lines that you've identified as
template<class _DataType>
struct SelectInternalConvertableToVectorType
{
typedef typename DataTypeTrait<_DataType>::TAbout TAbout;
typedef TL::IF<TL::MemberOf<TAbout,EnumDTP::Internal>::value>
TInternal;
typedef TL::IF<
TL::MemberOf<TAbout,EnumDTP::HasVectorType>::value
TConvertableToVectorType;
typedef TL::IF<
(TInternal::value &&
TConvertableToVectorType::value),_DataType,TL::False
::Result Result;
}; <------------------------ Appears in column 1
However, that "};" for me appeared in the correct column, not in column
0.
I wrote the buffer, started a 3rd emacs, went to the line, hit TAB and the
indentation
That, at least, is refreshing.
}; <------------------------ Correct.
Back in the original session TAB still places the '{' in column 1.
It doesn't for me. But I _have_ seen this error happening.

So, could you please try and give me a precise recipe for how to
reproduce the error, down at the level of what the buffer should look
like before inserting these lines, and which keys to press to insert
them. Was I correct in guessing that you typed in the lines when you
said you "added" them, as opposed to, say, C-y?
I am now attempting to try your c-state-cache-init and find this is not
defined.
That's exactly

M-: (c-state-cache-init)

, including the parens round the word. Are you certain you typed it
correctly? It should be there in Emacs 23.1.91.
Please advise what to do. I've attached "describe-bindings" if that helps.
Thanks,
Bob
PS "C-c C-b" runs "c-submit-bug-report" - is that what you expect?
Yes. It produces a dump of CC Mode's config as part of a mail buffer.
Cut and paste it into your own mail client.
--
Alan Mackenzie (Nuremberg, Germany).
Bob
2010-02-16 18:35:37 UTC
Permalink
Alan,
Post by Alan Mackenzie
I am now attempting to try your c-state-cache-init and find this is not
defined.
That's exactly
M-: (c-state-cache-init)
Ahh, typed it in wrong. I have been saving the buffer where the "
SelectInternalConvertableToVectorType" snippet is with the problematic '{'
in column 0.

So I am now able to eval your "(c-state-cache-init)" in the buffer.

The result is no difference, '{' still appears in column 0 when I TAB.
*Messages*
contains "1 (#o1, #x1)" after running it, if that means anything to you.

With regard to being more specific about what I type, that's going to be
really difficult.
Typically, I do a lot of kill - yanks, keyboard entry, and often return to
text to post-insert missing ','s or '>'s and then TAB or C-q C-j. That's
when I discover a bad indentation case. Sometimes, by TAB-ing on previous
lines then returning to the problematic line, the indentation corrects. I
have done what you did with the snippet also - put it into a separate file,
start a separate emacs session. Each time I do that, the indentation
corrects ('{' is not in col 0) and doesn't get out of whack as I fiddle in
it. Makes me think the problem occurs with more than 1 file visited.

I am preserving the emacs session with the bad '{' indentation (col 0). Is
there any way we can narrow the issue by probing the buffer, such as with
your (c-state-cache-init) ?

Attached is the "c-submit-bug-report" run from the problematic buffer.

Bob

-----Original Message-----
From: Alan Mackenzie [mailto:***@muc.de]
Sent: Tuesday, February 16, 2010 8:39 AM
To: Bob
Cc: 'Chong Yidong'; ***@debbugs.gnu.org
Subject: Re: bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting
issue.

Hi, Bob,
Post by Alan Mackenzie
Alan,
I am now running two emacs 23.1.91.1, one I started last week with many
files open in which I "work" etc., and one I just started with dribbling
"on" to edit a file I have seen the issue in. After half-hours editing in
this one, nothing bad has happened. I temporarily jumped back to the "work"
session to add a few needed lines and wammy! the problem appeared after
Could you be a little bit more exact in exactly what you did here. I
have just fired up emacs 23.1.91, typed in

namespace foo
{

(in column 0), then typed in the the lines that you've identified as
Post by Alan Mackenzie
template<class _DataType>
struct SelectInternalConvertableToVectorType
{
typedef typename DataTypeTrait<_DataType>::TAbout TAbout;
typedef TL::IF<TL::MemberOf<TAbout,EnumDTP::Internal>::value>
TInternal;
typedef TL::IF<
TL::MemberOf<TAbout,EnumDTP::HasVectorType>::value
TConvertableToVectorType;
typedef TL::IF<
(TInternal::value &&
TConvertableToVectorType::value),_DataType,TL::False
::Result Result;
}; <------------------------ Appears in column 1
However, that "};" for me appeared in the correct column, not in column
0.
Post by Alan Mackenzie
I wrote the buffer, started a 3rd emacs, went to the line, hit TAB and the
indentation
That, at least, is refreshing.
Post by Alan Mackenzie
}; <------------------------ Correct.
Back in the original session TAB still places the '{' in column 1.
It doesn't for me. But I _have_ seen this error happening.

So, could you please try and give me a precise recipe for how to
reproduce the error, down at the level of what the buffer should look
like before inserting these lines, and which keys to press to insert
them. Was I correct in guessing that you typed in the lines when you
said you "added" them, as opposed to, say, C-y?
Post by Alan Mackenzie
I am now attempting to try your c-state-cache-init and find this is not
defined.
That's exactly

M-: (c-state-cache-init)

, including the parens round the word. Are you certain you typed it
correctly? It should be there in Emacs 23.1.91.
Post by Alan Mackenzie
Please advise what to do. I've attached "describe-bindings" if that helps.
Thanks,
Bob
PS "C-c C-b" runs "c-submit-bug-report" - is that what you expect?
Yes. It produces a dump of CC Mode's config as part of a mail buffer.
Cut and paste it into your own mail client.
--
Alan Mackenzie (Nuremberg, Germany).
Alan Mackenzie
2010-02-17 09:46:52 UTC
Permalink
Hi, Bob,
Post by Bob
Alan,
Post by Alan Mackenzie
I am now attempting to try your c-state-cache-init and find this is not
defined.
That's exactly
M-: (c-state-cache-init)
Ahh, typed it in wrong. I have been saving the buffer where the "
SelectInternalConvertableToVectorType" snippet is with the problematic '{'
in column 0.
So I am now able to eval your "(c-state-cache-init)" in the buffer.
The result is no difference, '{' still appears in column 0 when I TAB.
*Messages*
contains "1 (#o1, #x1)" after running it, if that means anything to you.
Yes, thanks. The "1 .." is just random noise. The fact that
`c-state-cache-init' fixes nothing tells me that the bug's not to do with
a massive overhaul of the "state cache" (a cache of brace/paren/bracket
positions) I committed several months ago.
Post by Bob
With regard to being more specific about what I type, that's going to be
really difficult.
OK, I understand that.
Post by Bob
Typically, I do a lot of kill - yanks, keyboard entry, and often return
to text to post-insert missing ','s or '>'s and then TAB or C-q C-j.
That's when I discover a bad indentation case. Sometimes, by TAB-ing on
previous lines then returning to the problematic line, the indentation
corrects. I have done what you did with the snippet also - put it into
a separate file, start a separate emacs session. Each time I do that,
the indentation corrects ('{' is not in col 0) and doesn't get out of
whack as I fiddle in it. Makes me think the problem occurs with more
than 1 file visited.
I have a hyptothesis: noramlly, when you type in a < or > which "looks
like" a template delimiter, CC Mode marks it as an openining/closing
parenthesis. These can, and do, get confused with "proper" parens, { } (
) [ ], sometimes. It seems likely that on certain deletions, these
marked <s and >s become unbalanced, and this confuses the indentation.

I committed a fix for this on 2010-02-04, barely 2 weeks ago. This fix
ensures that these marks only get applied to < and > in balanced pairs,
and when stuff is deleted does the Right Thing. The fix hasn't yet found
its way into a pretest. If you test on the bzr head, could you try it
out, please. Otherwise, please be patient and try it out on the NEXT
pretest version, when that gets released.
Post by Bob
I am preserving the emacs session with the bad '{' indentation (col 0).
Is there any way we can narrow the issue by probing the buffer, such as
with your (c-state-cache-init) ?
Could you perhaps test the above hypothesis, please? Turn on
`show-paren-mode'. When you next hit this bad state in a buffer, place
point on a < or >, and see if it mismatches with a proper paren/brace.
Post by Bob
Attached is the "c-submit-bug-report" run from the problematic buffer.
Much appreciated.
Post by Bob
Bob
--
Alan Mackenzie (Nuremberg, Germany).
Bob
2010-02-17 16:40:36 UTC
Permalink
Alan,

Attached is "Foo.cpp" and "dribble" (if I did it correctly, dribble recorded
what I typed).

It illustrates a case where indentation is one level deeper than it should
be. (The other case we've been working on is one level shallower than it
should be). Maybe they are related?

visit-file "Foo.cpp", note indentation for "class Bar" is too deep. Now, TAB
on the "class Bar" line the indentation corrects. If you reconstruct the
file from "dribble" you should see TAB on the "class Bar" line incorrectly
indents too deep.

My ".emacs" file sets indentation to 2 spaces. I would guess there is an
issue where
the default (I think 4 spaces) is being used instead of my setting.

I've also attached my ".emacs" file as "emacs_init" (Outlook doesn't attach
files without a file-name). Please excuse the accumulation of junk in it
from many years of emacs use and
recommendations obtained from internet sites.

Bob

Foo.cpp:
----------------------------------------
template<
class Bar,
int _depth = -1
class Foo;

template<
int _depth = -1
class Foo
{
};
-----------------------------------------
Alan Mackenzie
2010-02-18 15:12:50 UTC
Permalink
Hi, Bob,
Post by Bob
Alan,
Attached is "Foo.cpp" and "dribble" (if I did it correctly, dribble recorded
what I typed).
It illustrates a case where indentation is one level deeper than it should
be. (The other case we've been working on is one level shallower than it
should be). Maybe they are related?
I suspect not, but who knows? :-(

However, I can reproduce this bug; the dribble file and your CC Mode
configuration file were utterly essential to my doing so. Thanks!
Post by Bob
visit-file "Foo.cpp", note indentation for "class Bar" is too deep. Now, TAB
on the "class Bar" line the indentation corrects. If you reconstruct the
file from "dribble" you should see TAB on the "class Bar" line incorrectly
indents too deep.
Yes.
Post by Bob
My ".emacs" file sets indentation to 2 spaces. I would guess there is an
issue where
the default (I think 4 spaces) is being used instead of my setting.
No, that's not it. In the course of creating this file, as per your
dribble, some "text properties" (deep in the bowels of CC Mode) are being
spuriously applied to certain characters in the buffer. I'll track down
exactly which keystroke provoked this, and hopefully be able to fix it
sometime soonish.

Thanks for taking all this trouble to help me reproduce the bug.
Post by Bob
Bob
--
Alan Mackenzie (Nuremberg, Germany).
Bob
2010-02-18 15:53:34 UTC
Permalink
Alan,

That's good news! Please let me know when you get it fixed and I'd like to
try it out.

BTW, the indenting and paren matching is a world better than 21.2, esp. with
regard to template '<' and '>' characters. Good job!

Bob

-----Original Message-----
From: Alan Mackenzie [mailto:***@muc.de]
Sent: Thursday, February 18, 2010 7:13 AM
To: Bob
Cc: 'Chong Yidong'; ***@debbugs.gnu.org
Subject: Re: bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting
issue.

Hi, Bob,
Post by Bob
Alan,
Attached is "Foo.cpp" and "dribble" (if I did it correctly, dribble recorded
what I typed).
It illustrates a case where indentation is one level deeper than it should
be. (The other case we've been working on is one level shallower than it
should be). Maybe they are related?
I suspect not, but who knows? :-(

However, I can reproduce this bug; the dribble file and your CC Mode
configuration file were utterly essential to my doing so. Thanks!
Post by Bob
visit-file "Foo.cpp", note indentation for "class Bar" is too deep. Now, TAB
on the "class Bar" line the indentation corrects. If you reconstruct the
file from "dribble" you should see TAB on the "class Bar" line incorrectly
indents too deep.
Yes.
Post by Bob
My ".emacs" file sets indentation to 2 spaces. I would guess there is an
issue where
the default (I think 4 spaces) is being used instead of my setting.
No, that's not it. In the course of creating this file, as per your
dribble, some "text properties" (deep in the bowels of CC Mode) are being
spuriously applied to certain characters in the buffer. I'll track down
exactly which keystroke provoked this, and hopefully be able to fix it
sometime soonish.

Thanks for taking all this trouble to help me reproduce the bug.
Post by Bob
Bob
--
Alan Mackenzie (Nuremberg, Germany).
Alan Mackenzie
2010-02-19 20:34:37 UTC
Permalink
Hi, Bob,

I think I've got the problem nailed. And I now think you were right
when you suggested it's related to the other indentation problems.

Would you please apply this patch to cc-mode.el, and tell me if the
problem's been fixed.

Thanks!


*** orig/cc-mode.el 2010-02-18 16:29:49.000000000 +0000
--- cc-mode.el 2010-02-19 20:15:14.630249504 +0000
***************
*** 635,641 ****
(save-excursion
(if c-get-state-before-change-functions
(mapc (lambda (fn)
! (funcall fn beg end))
c-get-state-before-change-functions))
(if c-before-font-lock-function
(funcall c-before-font-lock-function (point-min) (point-max)
--- 635,641 ----
(save-excursion
(if c-get-state-before-change-functions
(mapc (lambda (fn)
! (funcall fn (point-min) (point-max)))
c-get-state-before-change-functions))
(if c-before-font-lock-function
(funcall c-before-font-lock-function (point-min) (point-max)
***************
*** 1031,1036 ****
--- 1031,1044 ----
(when (> beg end)
(setq beg end)))

+ ;; C-y is capable of spuriously converting category properties
+ ;; c-</>-as-paren-syntax into hard syntax-table properties. Remove
+ ;; these when it happens.
+ (c-clear-char-property-with-value beg end 'syntax-table
+ c-<-as-paren-syntax)
+ (c-clear-char-property-with-value beg end 'syntax-table
+ c->-as-paren-syntax)
+
(c-trim-found-types beg end old-len) ; maybe we don't need all of these.
(c-invalidate-sws-region-after beg end)
(c-invalidate-state-cache beg)
Post by Bob
Alan,
That's good news! Please let me know when you get it fixed and I'd like to
try it out.
BTW, the indenting and paren matching is a world better than 21.2, esp. with
regard to template '<' and '>' characters. Good job!
Bob
--
Alan Mackenzie (Nuremberg, Germany).
Paul Pogonyshev
2010-02-20 13:41:02 UTC
Permalink
Severity: important
Post by Bob
Has anyone else seen this issue?
I see it so often it is not fun. I'm currently compiling an older
Emacs version because indentation's becoming broken every second line
is unbearable.

Paul
Paul Pogonyshev
2010-02-21 21:09:42 UTC
Permalink
Post by Paul Pogonyshev
Severity: important
Post by Bob
Has anyone else seen this issue?
I see it so often it is not fun. I'm currently compiling an older
Emacs version because indentation's becoming broken every second line
is unbearable.
Apologies. Please bear with me on this. It should become stable well
before the release of Emacs 23.2.
Is there any chance you could say the sort of editing you've been doing
which provokes this mis-indentation? Do you use C-y a lot? Is it only
in C++ Mode, or are other modes (C, Java, ....) affected?
Mostly in C++ Mode, with usually with quite complicated code
(templates), but not only. E.g. I lost patience with it on a file
without templates where only more or less non-common thing was a local
namespace. It looks basically like this (using my preferred
indentation style, might be important):

namespace
{
class Foo : public Bar
{
public:

virtual void
bla_bla (some_type& x, const string& y)
{
...
}

...
};
}

It used to break heavily in such functions (there are many similar)
inside a local class. Function names were becoming colored with
'font-lock-variable-name-face' and as such indented wrongly.
Any chance you might try the head versions in the bzr repository?
OK, but a bit later. I guess it is of little help, but currently I
use revision 98816 (just chose one at random) and cc-mode works fine.

Paul
Chong Yidong
2010-02-21 21:49:43 UTC
Permalink
Post by Paul Pogonyshev
Any chance you might try the head versions in the bzr repository?
OK, but a bit later. I guess it is of little help, but currently I
use revision 98816 (just chose one at random) and cc-mode works fine.
It would be nice if you could test the patch Alan posted at

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5570#40

since "Bob" has not responded to the request to test it.
Chong Yidong
2010-02-21 22:06:17 UTC
Permalink
Post by Chong Yidong
Post by Paul Pogonyshev
Any chance you might try the head versions in the bzr repository?
OK, but a bit later. I guess it is of little help, but currently I
use revision 98816 (just chose one at random) and cc-mode works fine.
It would be nice if you could test the patch Alan posted at
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5570#40
since "Bob" has not responded to the request to test it.
It doesn't apply to trunk and I'd prefer not to guess how to merge.
Can you rediff it?
Here, try this:

*** lisp/progmodes/cc-mode.el 2010-02-06 13:01:43 +0000
--- lisp/progmodes/cc-mode.el 2010-02-21 22:05:23 +0000
***************
*** 642,652 ****
(widen)
(save-excursion
(if c-get-state-before-change-functions
! (let ((beg (point-min))
! (end (point-max)))
! (mapc (lambda (fn)
! (funcall fn beg end))
! c-get-state-before-change-functions)))
(if c-before-font-lock-function
(funcall c-before-font-lock-function (point-min) (point-max)
(- (point-max) (point-min))))))
--- 642,650 ----
(widen)
(save-excursion
(if c-get-state-before-change-functions
! (mapc (lambda (fn)
! (funcall fn (point-min) (point-max)))
! c-get-state-before-change-functions))
(if c-before-font-lock-function
(funcall c-before-font-lock-function (point-min) (point-max)
(- (point-max) (point-min))))))
***************
*** 1042,1047 ****
--- 1040,1053 ----
(when (> beg end)
(setq beg end)))

+ ;; C-y is capable of spuriously converting category properties
+ ;; c-</>-as-paren-syntax into hard syntax-table properties. Remove
+ ;; these when it happens.
+ (c-clear-char-property-with-value beg end 'syntax-table
+ c-<-as-paren-syntax)
+ (c-clear-char-property-with-value beg end 'syntax-table
+ c->-as-paren-syntax)
+
(c-trim-found-types beg end old-len) ; maybe we don't need all of these.
(c-invalidate-sws-region-after beg end)
(c-invalidate-state-cache beg)
Paul Pogonyshev
2010-02-22 15:33:14 UTC
Permalink
Post by Chong Yidong
Post by Paul Pogonyshev
Any chance you might try the head versions in the bzr repository?
OK, but a bit later. I guess it is of little help, but currently I
use revision 98816 (just chose one at random) and cc-mode works fine.
It would be nice if you could test the patch Alan posted at
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5570#40
since "Bob" has not responded to the request to test it.
It doesn't apply to trunk and I'd prefer not to guess how to merge.
Can you rediff it?
Here, try this: [...]
Thanks, indentation problems seem to be gone. Or at least don't
happen nearly that often.

Paul
Paul Pogonyshev
2010-02-23 13:28:08 UTC
Permalink
Post by Paul Pogonyshev
Thanks, indentation problems seem to be gone. Or at least don't
happen nearly that often.
Still happens, though much more rarely than before. No sure way to
reproduce, but in this short source code file:

#include "engine/strategy/evaluation/material.hpp"

#include <utility>


using namespace engine;

using namespace std;


int
MaterialEvaluationFunction::do_evaluate_material (Board::PackedMaterialData material)
{
const int score = _algorithm->evaluate (material);
const int normalized_score = ((score * get_first_rabbit_value ())
/ _algorithm->get_first_rabbit_value ());

_cache.insert (make_pair (material, normalized_score));
}

I'm getting wrongly indented '_cache.insert ...' line. C-c C-s on it
also shows 'topmost-intro'. I guess this heavily depends on the order
in which code is typed.

After I killed the buffer and reopened it, indentation is back to
normal and C-c C-s on that line correctly shows 'statement'.

Paul
Paul Pogonyshev
2010-02-21 21:54:23 UTC
Permalink
Post by Chong Yidong
Post by Paul Pogonyshev
Any chance you might try the head versions in the bzr repository?
OK, but a bit later. I guess it is of little help, but currently I
use revision 98816 (just chose one at random) and cc-mode works fine.
It would be nice if you could test the patch Alan posted at
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5570#40
since "Bob" has not responded to the request to test it.
It doesn't apply to trunk and I'd prefer not to guess how to merge.
Can you rediff it?

Paul
Alan Mackenzie
2010-02-21 21:01:36 UTC
Permalink
Hi, Paul,
Post by Paul Pogonyshev
Severity: important
Post by Bob
Has anyone else seen this issue?
I see it so often it is not fun. I'm currently compiling an older
Emacs version because indentation's becoming broken every second line
is unbearable.
Apologies. Please bear with me on this. It should become stable well
before the release of Emacs 23.2.

Is there any chance you could say the sort of editing you've been doing
which provokes this mis-indentation? Do you use C-y a lot? Is it only
in C++ Mode, or are other modes (C, Java, ....) affected?

Any chance you might try the head versions in the bzr repository?
Post by Paul Pogonyshev
Paul
--
Alan Mackenzie (Nuremberg, Germany).
Paul Pogonyshev
2010-03-18 20:05:11 UTC
Permalink
After March the 1st changes indentation in C++ and related modes is
better, but it still breaks often enough --- to the tune of once every
3--5 minutes or so. Both when editing C++ and Java code.

As I understand, the issue is that some cached information becomes
wrong after certain (undetermined) buffer changes. Maybe it is better
to write an automated cache validation code rather than search for a
certain way to reproduce stuff?

E.g. just build information from scratch and assert that cached
information is the same. Ideally, such checks should be performed
after every change, so that you'd be able to find which change exactly
trigerred a bug. (Granted, this would make debugging version slow as
hell, but with this rate of error occurence it's not like you'd need
to run it for days.)

Without automated error detection tools cc-mode is bound to have
similar issues after every second non-trivial change.

Paul
Chong Yidong
2010-03-18 20:31:05 UTC
Permalink
Post by Paul Pogonyshev
After March the 1st changes indentation in C++ and related modes is
better, but it still breaks often enough --- to the tune of once every
3--5 minutes or so. Both when editing C++ and Java code.
This is troubling.

If the CC-mode changes are still not quite stabilized, I would like to
consider postphoning them to post-23.2. Alan, what's your opinion?
Could we revert to the version in the branch prior to 2010-02-04 (which
IIUC is the change that introduced these problems), or do you consider
those changes too important for the release?
Alan Mackenzie
2010-03-19 21:56:39 UTC
Permalink
Hi, Paul,
Post by Paul Pogonyshev
After March the 1st changes indentation in C++ and related modes is
better, but it still breaks often enough --- to the tune of once every
3--5 minutes or so. Both when editing C++ and Java code.
How about plain C? Is the breakage connected with templates/generics, as
far as you can see?

What is the breakage like? Is it a one-off failure which repairs itself
after a few more edits, or is it more like a creeping death? Does
reinitializing the mode (M-x c++-mode) repair the indentation? Does
saving the file and reloading it fix things?

Does
M-x (c-state-cache-init)
fix it?
Post by Paul Pogonyshev
As I understand, the issue is that some cached information becomes
wrong after certain (undetermined) buffer changes.
This is very likely.
Post by Paul Pogonyshev
Maybe it is better to write an automated cache validation code rather
than search for a certain way to reproduce stuff?
For which cache, though? Hence the questions above.
Post by Paul Pogonyshev
E.g. just build information from scratch and assert that cached
information is the same. Ideally, such checks should be performed
after every change, so that you'd be able to find which change exactly
trigerred a bug. (Granted, this would make debugging version slow as
hell, but with this rate of error occurence it's not like you'd need
to run it for days.)
I should be able to get this done by, say, Monday for the "state-cache".
Again, does M-x (c-state-cache-init) clear the fault?
Post by Paul Pogonyshev
Without automated error detection tools cc-mode is bound to have
similar issues after every second non-trivial change.
I strongly believe it's a very small finite number of distinct bugs
causing the problems. Anything more you can tell me about how it breaks
would be of the utmost help.
Post by Paul Pogonyshev
Paul
--
Alan Mackenzie (Nuremberg, Germany).
Jan Djärv
2010-03-19 22:45:40 UTC
Permalink
Post by Alan Mackenzie
Hi, Paul,
Post by Paul Pogonyshev
After March the 1st changes indentation in C++ and related modes is
better, but it still breaks often enough --- to the tune of once every
3--5 minutes or so. Both when editing C++ and Java code.
How about plain C? Is the breakage connected with templates/generics, as
far as you can see?
FWIW, I see breakage in plain C also.
Post by Alan Mackenzie
What is the breakage like? Is it a one-off failure which repairs itself
after a few more edits, or is it more like a creeping death? Does
reinitializing the mode (M-x c++-mode) repair the indentation? Does
saving the file and reloading it fix things?
I now more or less automatically do M-x revert buffer when this happens.

Jan D.
Alan Mackenzie
2010-03-20 13:22:50 UTC
Permalink
Hi, Jan,
Post by Jan Djärv
Post by Alan Mackenzie
Hi, Paul,
Post by Paul Pogonyshev
After March the 1st changes indentation in C++ and related modes is
better, but it still breaks often enough --- to the tune of once every
3--5 minutes or so. Both when editing C++ and Java code.
How about plain C? Is the breakage connected with templates/generics, as
far as you can see?
FWIW, I see breakage in plain C also.
That's worth a great deal to know. It suggests that the changes of
2010-02-04 aren't responsble for this bug.
Post by Jan Djärv
Post by Alan Mackenzie
What is the breakage like? Is it a one-off failure which repairs itself
after a few more edits, or is it more like a creeping death? Does
reinitializing the mode (M-x c++-mode) repair the indentation? Does
saving the file and reloading it fix things?
I now more or less automatically do M-x revert buffer when this happens.
When _what_ happens? I take it, then, that revert-buffer fixes the bad
state, and you save the file first before reverting it. Any chance you
could answer the above questions, together with the further one:

Does
M-x (c-state-cache-init)
fix it?

These answers would be of the utmost help. Thanks!
Post by Jan Djärv
Jan D.
--
Alan Mackenzie (Nuremberg, Germany).
Jan D.
2010-03-20 14:49:07 UTC
Permalink
Post by Alan Mackenzie
Hi, Jan,
Post by Jan Djärv
Post by Alan Mackenzie
Hi, Paul,
Post by Paul Pogonyshev
After March the 1st changes indentation in C++ and related modes is
better, but it still breaks often enough --- to the tune of once every
3--5 minutes or so. Both when editing C++ and Java code.
How about plain C? Is the breakage connected with templates/generics, as
far as you can see?
FWIW, I see breakage in plain C also.
That's worth a great deal to know. It suggests that the changes of
2010-02-04 aren't responsble for this bug.
Post by Jan Djärv
Post by Alan Mackenzie
What is the breakage like? Is it a one-off failure which repairs itself
after a few more edits, or is it more like a creeping death? Does
reinitializing the mode (M-x c++-mode) repair the indentation? Does
saving the file and reloading it fix things?
I now more or less automatically do M-x revert buffer when this happens.
When _what_ happens?
When C-mode thinks everything is topmost-intro.
Post by Alan Mackenzie
I take it, then, that revert-buffer fixes the bad
state, and you save the file first before reverting it. Any chance you
Does
M-x (c-state-cache-init)
fix it?
No, it does not.

Jan D.
Alan Mackenzie
2010-03-21 11:16:27 UTC
Permalink
Hi, Jan
Post by Jan D.
Post by Alan Mackenzie
Post by Jan Djärv
Post by Alan Mackenzie
How about plain C? Is the breakage connected with
templates/generics, as far as you can see?
FWIW, I see breakage in plain C also.
That's worth a great deal to know. It suggests that the changes of
2010-02-04 aren't responsble for this bug.
Post by Jan Djärv
I now more or less automatically do M-x revert buffer when this happens.
When _what_ happens?
When C-mode thinks everything is topmost-intro.
OK. I get the impression this is VERY annoying, but that it happens very
frequently rather than just now and then. Good!
Post by Jan D.
Post by Alan Mackenzie
I take it, then, that revert-buffer fixes the bad
state, and you save the file first before reverting it. Any chance you
Does
M-x (c-state-cache-init)
fix it?
No, it does not.
I'm assuming your Emacs is pretty much the newest bzr version.

Would you please choose a C file, as small as will work, and start an
Emacs session with a dribble file (see page "Recording Input" in the
Elisp manual). Edit the C file until indentation goes wrong. Then send
me the dribble file, the (original) C file, and your CC Mode
configuration (generated by C-c C-b in the C file).

Apologies for this horrible bug, and thanks for you help in nailing it
down!
Post by Jan D.
Jan D.
--
Alan Mackenzie (Nuremberg, Germany).
Jan Djärv
2010-03-21 11:50:44 UTC
Permalink
Post by Alan Mackenzie
I'm assuming your Emacs is pretty much the newest bzr version.
Pretty much.
Post by Alan Mackenzie
Would you please choose a C file, as small as will work, and start an
Emacs session with a dribble file (see page "Recording Input" in the
Elisp manual). Edit the C file until indentation goes wrong.
It isn't that easy to reproduce, it just sort of happens.
I have no idea if it will happen on a small file.
Post by Alan Mackenzie
Then send
me the dribble file, the (original) C file, and your CC Mode
configuration (generated by C-c C-b in the C file).
I'll try.

Jan D.
Jan Djärv
2010-03-22 08:07:06 UTC
Permalink
Post by Alan Mackenzie
Hi, Jan
Post by Jan D.
Post by Alan Mackenzie
Does
M-x (c-state-cache-init)
fix it?
No, it does not.
I just got the problem again, while editing gtkutil.c in Emacs. There
M-: (c-state-cache-init) did indeed fix the problem.

So it sometimes help. I guess this only adds to the confusion...

Jan D.
Paul Pogonyshev
2010-04-02 17:01:36 UTC
Permalink
Post by Jan Djärv
Post by Jan D.
Post by Alan Mackenzie
Does
M-x (c-state-cache-init)
fix it?
No, it does not.
I just got the problem again, while editing gtkutil.c in Emacs. There
M-: (c-state-cache-init) did indeed fix the problem.
So it sometimes help. I guess this only adds to the confusion...
I guess that's because there are several similar bugs. At the rate
fixing goes (or rather doesn't go) I think I'll compile and use an old
Emacs without indentation bugs again.

Paul


P.S. By the way: maybe this change did fix the original bug, but it
didn't improve overall indentation problems in CC modes noticeably.

2010-03-24 Alan Mackenzie <***@muc.de>

* progmodes/cc-engine.el (c-remove-stale-state-cache):
Fix off-by-one error. Fixes bug #5747.
Paul Pogonyshev
2010-04-04 00:14:14 UTC
Permalink
Post by Jan Djärv
Post by Jan D.
Post by Alan Mackenzie
Does
M-x (c-state-cache-init)
fix it?
No, it does not.
I just got the problem again, while editing gtkutil.c in Emacs. There
M-: (c-state-cache-init) did indeed fix the problem.
So it sometimes help. I guess this only adds to the confusion...
You sent this message on March 22. That was the date Alan checked this
Fix off-by-one error. Fixes bug #5747.
The patch was merged to the trunk a couple days later. I don't know
whether you were running with it when you made the above observation.
So, are you still observing the problem after the patch?
Not directed to me, but I can answer as well.

Yes, I see the problem, though I'm not sure if it is "the" problem,
since there might be many similar ones.

* At least in some cases M-x (c-state-cache-init) doesn't help, but
C-x RET r (buffer reloading) does;

* Occasionally even buffer reloading doesn't help, so I have to kill
the buffer and open the file again;

* Sometimes I get wrong indentation which miraculously fixes itself
after I hit TAB next time;

* Sometimes when I type in a C string (like "bla-bla-bla"), first
character after the opening quote gets eaten because of a Lisp
error. Will report as separate bug when it happens if I don't
forget.

And yes, I double-verified that I do have the fix in the version I'm
running now.

Paul
Chong Yidong
2010-04-03 23:55:53 UTC
Permalink
Post by Jan Djärv
Post by Jan D.
Post by Alan Mackenzie
Does
M-x (c-state-cache-init)
fix it?
No, it does not.
I just got the problem again, while editing gtkutil.c in Emacs. There
M-: (c-state-cache-init) did indeed fix the problem.
So it sometimes help. I guess this only adds to the confusion...
You sent this message on March 22. That was the date Alan checked this
into the branch:

2010-03-22 Alan Mackenzie <***@muc.de>

* progmodes/cc-engine.el (c-remove-stale-state-cache):
Fix off-by-one error. Fixes bug #5747.

The patch was merged to the trunk a couple days later. I don't know
whether you were running with it when you made the above observation.

So, are you still observing the problem after the patch?
Jan Djärv
2010-04-04 08:09:25 UTC
Permalink
Post by Jan Djärv
Post by Jan D.
Post by Alan Mackenzie
Does
M-x (c-state-cache-init)
fix it?
No, it does not.
I just got the problem again, while editing gtkutil.c in Emacs. There
M-: (c-state-cache-init) did indeed fix the problem.
So it sometimes help. I guess this only adds to the confusion...
You sent this message on March 22. That was the date Alan checked this
Fix off-by-one error. Fixes bug #5747.
The patch was merged to the trunk a couple days later. I don't know
whether you were running with it when you made the above observation
I was running without it.
.
So, are you still observing the problem after the patch?
Yes, but it seems to happen less frequently. Also, sometimes just another TAB
at BOL fixes it.

Jan D.
Chong Yidong
2010-04-28 18:55:34 UTC
Permalink
Hi Alan,

I found a small recipe for reproducing a (the?) cache bug on the trunk:

1. emacs -Q

2. C-x C-f foo.c RET

3. Type the following (from view-lossage):

i n t SPC m a i n SPC ( ) <return>
{ <return> f o o ; <return> } M-< <return>
<up> # i n c l u d e SPC < s t d i o . h > <return>
<down> <down> <down> <tab>

At this point, the line "foo;" becomes indented to the zeroth column.

Does this help?
Alan Mackenzie
2010-04-29 15:04:42 UTC
Permalink
Hi, Yidong and Paul!
Post by Chong Yidong
Hi Alan,
1. emacs -Q
2. C-x C-f foo.c RET
i n t SPC m a i n SPC ( ) <return>
{ <return> f o o ; <return> } M-< <return>
<up> # i n c l u d e SPC < s t d i o . h > <return>
<down> <down> <down> <tab>
At this point, the line "foo;" becomes indented to the zeroth column.
Does this help?
Does it help? That was an absolute dream of a bug report. :-)

The problem was another off-by-one error. What triggered the bug is
typing <return> at the end of the #include line. The thing marking the
end of the macro (a category text property) got pushed onto the next line
instead of being properly removed. This then fouled up the
"c-state-cache" mechanism.

It looks like the situation reported by Paul (in his mail of 2010-02-23,
Message-Id: <***@gmx.net>) is likely caused by
the same bug.

I've committed a fix to the Bazaar HEAD, which of course is a fix for the
last bug in this area. ;-)

Paul, would you please update your copy of the Emacs HEAD from savannah,
and see if any indentation bugs still manifest themselves. Thanks!
--
Alan Mackenzie (Nuremberg, Germany).
Paul Pogonyshev
2010-05-29 22:27:47 UTC
Permalink
Post by Alan Mackenzie
Paul, would you please update your copy of the Emacs HEAD from savannah,
and see if any indentation bugs still manifest themselves. Thanks!
Yes they do. I still see all kinds of indentation bugs, none seem to
be gone:

* Sometimes everything is topmost-intro; this may or may not be gone
after buffer reload with C-x RET r, suggesting at least two sources
for the bug.

* Indentation sometimes spuriosly changes when hitting TAB (e.g. TAB
-> to 0th column, TAB again -> to 8th column, TAB third time -> to
0th once more; no intervening keystrokes).

* Sometimes character after opening " gets eaten see #5851.

I did 'bzr pull', then 'make maintainer-clean && ./configure && make'
and then 'sudo make install'.

Paul

Loading...