Discussion:
[Swig-devel] swig 3.0.5 wrap file size tripled than 2.0.1
Yong Chen
2015-06-10 18:51:21 UTC
Permalink
Hi,

I recently upgraded swig from 2.0.1 to 3.0.5, and found the size of
my_file_wrap.cpp tripled, especially there's lots of new "_swigconstant"
related functions added. This caused my gcc to fail.

I wonder if it's the expected behavior, or there is some flags that I can
use to reduce the file size?

thanks,
Yong Chen
Paweł Tomulik
2015-06-10 21:25:12 UTC
Permalink
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5, and found the size of
my_file_wrap.cpp tripled, especially there's lots of new "_swigconstant"
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there is some flags that I
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Hi,

these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and provide a way to implement
object constants (constants of type T where T is class/struct). From
what I see (https://github.com/swig/swig/pull/250/files) using -builtin
swig flag should help you to get rid of these "_swigconstnants" (the
constant should then be mostly handled in the old way).
--
Pawel Tomulik

------------------------------------------------------------------------------
Yong Chen
2015-06-11 18:57:33 UTC
Permalink
Thanks Pawel.

After adding "-builtin" flag to swig, the size of the generated file
my_file_wrap.cpp reduced from 95MB to 25MB (I know it's still big). But it
caused gcc to fail, if I have gcc flag "-O1" set, "-O0" works fine.

Do you know if it's a known issue?

Thanks,
Yong Chen

gcc: Internal error: Segmentation fault (program cc1plus)
Please submit a full bug report.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
error: command 'gcc' failed with exit status 1
Post by Yong Chen
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5, and found the size of
my_file_wrap.cpp tripled, especially there's lots of new "_swigconstant"
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there is some flags that I
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Hi,
these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and provide a way to implement
object constants (constants of type T where T is class/struct). From
what I see (https://github.com/swig/swig/pull/250/files) using -builtin
swig flag should help you to get rid of these "_swigconstnants" (the
constant should then be mostly handled in the old way).
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Paweł Tomulik
2015-06-11 19:42:00 UTC
Permalink
Well,

I bet you're running out of memory - 25MB is a huge amount of code.
In my project largest wrapper is about 1MB and it's almost 25.000 lines
(so I suppose your shall have more than 500.000, that's a lot). I
wonder, what is that thing you're trying to wrap? Maybe it's worth to
split it up into pieces?



Regards
Post by Yong Chen
Thanks Pawel.
After adding "-builtin" flag to swig, the size of the generated file
my_file_wrap.cpp reduced from 95MB to 25MB (I know it's still big). But
it caused gcc to fail, if I have gcc flag "-O1" set, "-O0" works fine.
Do you know if it's a known issue?
Thanks,
Yong Chen
gcc: Internal error: Segmentation fault (program cc1plus)
Please submit a full bug report.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
error: command 'gcc' failed with exit status 1
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5, and found the size of
my_file_wrap.cpp tripled, especially there's lots of new
"_swigconstant"
Post by Yong Chen
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there is some flags that I
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Hi,
these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and provide a way to implement
object constants (constants of type T where T is class/struct). From
what I see (https://github.com/swig/swig/pull/250/files) using -builtin
swig flag should help you to get rid of these "_swigconstnants" (the
constant should then be mostly handled in the old way).
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik

------------------------------------------------------------------------------
Paweł Tomulik
2015-06-11 20:20:03 UTC
Permalink
It may be simply a gcc bug. Have you tried different gcc versions?
I monitored the virtual memory usage during gcc compilation, it's about
2GB, less than the 4GB address space.
It's 3rd party .h files that we are trying to wrap. Splitting file is
definitely a good way, another way is to do "gcc -O0", instead of "gcc
-O1". But I'm told to at least find out why gcc crashes.
Filtering out the unused constants (on Python side) is also a way that I
will take a look.
Thanks for your quick response,
Yong Chen
Well,
I bet you're running out of memory - 25MB is a huge amount of code.
In my project largest wrapper is about 1MB and it's almost 25.000 lines
(so I suppose your shall have more than 500.000, that's a lot). I
wonder, what is that thing you're trying to wrap? Maybe it's worth to
split it up into pieces?
Regards
Post by Yong Chen
Thanks Pawel.
After adding "-builtin" flag to swig, the size of the generated file
my_file_wrap.cpp reduced from 95MB to 25MB (I know it's still big). But
it caused gcc to fail, if I have gcc flag "-O1" set, "-O0" works fine.
Do you know if it's a known issue?
Thanks,
Yong Chen
gcc: Internal error: Segmentation fault (program cc1plus)
Please submit a full bug report.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
error: command 'gcc' failed with exit status 1
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5, and found the size of
my_file_wrap.cpp tripled, especially there's lots of new
"_swigconstant"
Post by Yong Chen
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there is some flags that I
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Hi,
these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and provide a way to implement
object constants (constants of type T where T is class/struct). From
what I see (https://github.com/swig/swig/pull/250/files) using -builtin
swig flag should help you to get rid of these "_swigconstnants" (the
constant should then be mostly handled in the old way).
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik

------------------------------------------------------------------------------
Yong Chen
2015-06-12 18:20:50 UTC
Permalink
Yes possible, changing from -O1 to -O0 makes the problem go away.
Unfortunately for me it's not easy to change gcc version due to other
reasons.

Is specifying "-builtin" the only way to not generate those
"_swigconstants"? I see "-builtin" is a new flag, wondering if I can avoid
to use it, but still not generate the "_swigconstants".
Post by Paweł Tomulik
It may be simply a gcc bug. Have you tried different gcc versions?
I monitored the virtual memory usage during gcc compilation, it's about
2GB, less than the 4GB address space.
It's 3rd party .h files that we are trying to wrap. Splitting file is
definitely a good way, another way is to do "gcc -O0", instead of "gcc
-O1". But I'm told to at least find out why gcc crashes.
Filtering out the unused constants (on Python side) is also a way that I
will take a look.
Thanks for your quick response,
Yong Chen
Well,
I bet you're running out of memory - 25MB is a huge amount of code.
In my project largest wrapper is about 1MB and it's almost 25.000
lines
(so I suppose your shall have more than 500.000, that's a lot). I
wonder, what is that thing you're trying to wrap? Maybe it's worth to
split it up into pieces?
Regards
Post by Yong Chen
Thanks Pawel.
After adding "-builtin" flag to swig, the size of the generated
file
Post by Yong Chen
my_file_wrap.cpp reduced from 95MB to 25MB (I know it's still
big). But
Post by Yong Chen
it caused gcc to fail, if I have gcc flag "-O1" set, "-O0" works
fine.
Post by Yong Chen
Do you know if it's a known issue?
Thanks,
Yong Chen
gcc: Internal error: Segmentation fault (program cc1plus)
Please submit a full bug report.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
error: command 'gcc' failed with exit status 1
On Wed, Jun 10, 2015 at 2:25 PM, Paweł Tomulik <
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5, and found the
size of
Post by Yong Chen
Post by Yong Chen
my_file_wrap.cpp tripled, especially there's lots of new
"_swigconstant"
Post by Yong Chen
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there is some
flags that I
Post by Yong Chen
Post by Yong Chen
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
Post by Yong Chen
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Hi,
these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and provide a way to
implement
Post by Yong Chen
object constants (constants of type T where T is
class/struct). From
Post by Yong Chen
what I see (https://github.com/swig/swig/pull/250/files)
using -builtin
Post by Yong Chen
swig flag should help you to get rid of these
"_swigconstnants" (the
Post by Yong Chen
constant should then be mostly handled in the old way).
--
Pawel Tomulik
------------------------------------------------------------------------------
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Paweł Tomulik
2015-06-13 06:20:37 UTC
Permalink
Post by Yong Chen
Yes possible, changing from -O1 to -O0 makes the problem go away.
Unfortunately for me it's not easy to change gcc version due to other
reasons.
Why? Can't you just get your generated *.cc wrapper and try to compile
somewhere. There are even online services offering multiple versions of
gcc at once so it's just a matter of uploading your file and trying to
compile (ok, size might be a problem, but anyway it's worth to try at
least).
Post by Yong Chen
Is specifying "-builtin" the only way to not generate those
"_swigconstants"? I see "-builtin" is a new flag, wondering if I can
avoid to use it, but still not generate the "_swigconstants".
I don't think it's so new. It seem to become quite standard, I have even
seen a discussion where it has been considered to be enabled by default
in future versions.

*_swigconstant are generated for each constant found in the wrapped
interface file (.h header in your case); if I recall correctly it
pertains to both - variables defined as "const" and preprocessor macros.
Eliminating (e.g. %ignoring) unused (or all) macros/constants may help
(I suppose, a 3rd party header file may contain quite a lot). Take a
look at
http://swig.org/Doc3.0/SWIGDocumentation.html#SWIG_limiting_renaming to
see how to ignore given category of compounds

It may help reducing the size of your generated wrapper in general (with
and without -builtin).
Post by Yong Chen
It may be simply a gcc bug. Have you tried different gcc versions?
I monitored the virtual memory usage during gcc compilation, it's about
2GB, less than the 4GB address space.
It's 3rd party .h files that we are trying to wrap. Splitting file is
definitely a good way, another way is to do "gcc -O0", instead of "gcc
-O1". But I'm told to at least find out why gcc crashes.
Filtering out the unused constants (on Python side) is also a way that I
will take a look.
Thanks for your quick response,
Yong Chen
Well,
I bet you're running out of memory - 25MB is a huge amount of code.
In my project largest wrapper is about 1MB and it's almost 25.000 lines
(so I suppose your shall have more than 500.000, that's a lot). I
wonder, what is that thing you're trying to wrap? Maybe it's worth to
split it up into pieces?
Regards
Post by Yong Chen
Thanks Pawel.
After adding "-builtin" flag to swig, the size of the generated file
my_file_wrap.cpp reduced from 95MB to 25MB (I know it's still big). But
it caused gcc to fail, if I have gcc flag "-O1" set, "-O0" works fine.
Do you know if it's a known issue?
Thanks,
Yong Chen
gcc: Internal error: Segmentation fault (program cc1plus)
Please submit a full bug report.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
error: command 'gcc' failed with exit status 1
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5, and found the size of
my_file_wrap.cpp tripled, especially there's lots of new
"_swigconstant"
Post by Yong Chen
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there is some flags that I
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Hi,
these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and provide a way to implement
object constants (constants of type T where T is class/struct). From
what I see (https://github.com/swig/swig/pull/250/files) using -builtin
swig flag should help you to get rid of these "_swigconstnants" (the
constant should then be mostly handled in the old way).
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik

------------------------------------------------------------------------------
Paweł Tomulik
2015-06-13 09:02:25 UTC
Permalink
%rename may be used to ignore stuff per category: functions, variables,
classes, etc.. I suppose it's possible to ignore constants similarly.
It's not exemplified in the documentation, but (as mentioned in the
manual) you may take a look into swig.swg of your swig installation (on
my Debian it's /usr/share/swig/3.0.5/swig.swg) to see what %$isxxxx and
%$hasxxxx macros are defined. In your case it's probably %$isconstant, so

%rename($ignore, "%$isconstant") ""; // Only ignore all constants

should do the job. At least for the first trial.
Thanks Pawel, for your reply.
I installed gcc 4.9 and seeing the same segmentation fault (as gcc 4.5).
I checked the link you gave on not generating the _swigconstants. Seems
to use %ignore, it needs to know what to ignore (such as name, for
match). I wonder if there is a way totally skip it. (I know -builtin
would be a good method, unfortunately it causes gcc to crash when
compiling the wrapper file).
Post by Yong Chen
Yes possible, changing from -O1 to -O0 makes the problem go away.
Unfortunately for me it's not easy to change gcc version due to other
reasons.
Why? Can't you just get your generated *.cc wrapper and try to compile
somewhere. There are even online services offering multiple versions of
gcc at once so it's just a matter of uploading your file and trying to
compile (ok, size might be a problem, but anyway it's worth to try at
least).
Post by Yong Chen
Is specifying "-builtin" the only way to not generate those
"_swigconstants"? I see "-builtin" is a new flag, wondering if I can
avoid to use it, but still not generate the "_swigconstants".
I don't think it's so new. It seem to become quite standard, I have even
seen a discussion where it has been considered to be enabled by default
in future versions.
*_swigconstant are generated for each constant found in the wrapped
interface file (.h header in your case); if I recall correctly it
pertains to both - variables defined as "const" and preprocessor macros.
Eliminating (e.g. %ignoring) unused (or all) macros/constants may help
(I suppose, a 3rd party header file may contain quite a lot). Take a
look at
http://swig.org/Doc3.0/SWIGDocumentation.html#SWIG_limiting_renaming to
see how to ignore given category of compounds
It may help reducing the size of your generated wrapper in general (with
and without -builtin).
Post by Yong Chen
It may be simply a gcc bug. Have you tried different gcc versions?
I monitored the virtual memory usage during gcc compilation, it's about
2GB, less than the 4GB address space.
It's 3rd party .h files that we are trying to wrap. Splitting file is
definitely a good way, another way is to do "gcc -O0", instead of "gcc
-O1". But I'm told to at least find out why gcc crashes.
Filtering out the unused constants (on Python side) is also a way that I
will take a look.
Thanks for your quick response,
Yong Chen
Well,
I bet you're running out of memory - 25MB is a huge
amount of code.
Post by Yong Chen
In my project largest wrapper is about 1MB and it's
almost 25.000 lines
Post by Yong Chen
(so I suppose your shall have more than 500.000, that's
a lot). I
Post by Yong Chen
wonder, what is that thing you're trying to wrap? Maybe
it's worth to
Post by Yong Chen
split it up into pieces?
Regards
Post by Yong Chen
Thanks Pawel.
After adding "-builtin" flag to swig, the size of the
generated file
Post by Yong Chen
Post by Yong Chen
my_file_wrap.cpp reduced from 95MB to 25MB (I know
it's still big). But
Post by Yong Chen
Post by Yong Chen
it caused gcc to fail, if I have gcc flag "-O1" set,
"-O0" works fine.
Post by Yong Chen
Post by Yong Chen
Do you know if it's a known issue?
Thanks,
Yong Chen
gcc: Internal error: Segmentation fault (program cc1plus)
Please submit a full bug report.
See <http://bugzilla.redhat.com/bugzilla> for
instructions.
Post by Yong Chen
Post by Yong Chen
error: command 'gcc' failed with exit status 1
On Wed, Jun 10, 2015 at 2:25 PM, Paweł Tomulik
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5,
and found the size of
Post by Yong Chen
Post by Yong Chen
Post by Yong Chen
my_file_wrap.cpp tripled, especially there's
lots of new
Post by Yong Chen
Post by Yong Chen
"_swigconstant"
Post by Yong Chen
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there
is some flags that I
Post by Yong Chen
Post by Yong Chen
Post by Yong Chen
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
Post by Yong Chen
Post by Yong Chen
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Post by Yong Chen
Post by Yong Chen
Hi,
these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and
provide a way to implement
Post by Yong Chen
Post by Yong Chen
object constants (constants of type T where T is
class/struct). From
Post by Yong Chen
Post by Yong Chen
what I see
(https://github.com/swig/swig/pull/250/files) using -builtin
Post by Yong Chen
Post by Yong Chen
swig flag should help you to get rid of these
"_swigconstnants" (the
Post by Yong Chen
Post by Yong Chen
constant should then be mostly handled in the old
way).
Post by Yong Chen
Post by Yong Chen
--
Pawel Tomulik
------------------------------------------------------------------------------
Post by Yong Chen
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Post by Yong Chen
--
Pawel Tomulik
------------------------------------------------------------------------------
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik
------------------------------------------------------------------------------
Post by Yong Chen
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
--
Pawel Tomulik
--
Pawel Tomulik

------------------------------------------------------------------------------
Paweł Tomulik
2015-06-11 19:39:35 UTC
Permalink
Anyway, this suggests that you probably have a lot of constants
(defines?) in the source file you're wrapping. Are they used in the
Python side? Maybe you should consider ignoring them somehow?
Post by Paweł Tomulik
Post by Yong Chen
Hi,
I recently upgraded swig from 2.0.1 to 3.0.5, and found the size of
my_file_wrap.cpp tripled, especially there's lots of new "_swigconstant"
related functions added. This caused my gcc to fail.
I wonder if it's the expected behavior, or there is some flags that I
can use to reduce the file size?
thanks,
Yong Chen
------------------------------------------------------------------------------
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Hi,
these "_swigconstant's" were introduced in PR 250
(https://github.com/swig/swig/pull/250), and provide a way to implement
object constants (constants of type T where T is class/struct). From
what I see (https://github.com/swig/swig/pull/250/files) using -builtin
swig flag should help you to get rid of these "_swigconstnants" (the
constant should then be mostly handled in the old way).
--
Pawel Tomulik

------------------------------------------------------------------------------
Loading...