Discussion:
[Swig-devel] Should new code call Delete() at all?
Vadim Zeitlin
2016-05-25 22:17:12 UTC
Permalink
Hello,

Currently there is an enormous number of memory leaks in SWIG code as
plenty of DOH objects are never freed. It could be argued that it doesn't
really matter for a program such as SWIG which doesn't run for a long time
and so it doesn't matter if it leaks memory anyhow as all of it will be
reclaimed by the OS on process termination. However the problem is that
there are still _some_ calls to Delete() in SWIG code. And I never know if
I should care about memory being leaked and try to at least call Delete()
correctly in the new code or not.

To make this more concrete, I'm trying merging the latest master into the
doxygen branch. There are plenty of non-trivial conflicts as handling of
docstring indentation in master was completely modified in the meanwhile
(commit fa282b3540c87927fc3562bd0f0863accf6a853f and others), so I have to
change some code manually. My existing code in the branch tries to avoid
memory leaks but the new code doesn't, at all, e.g. this:

Printv(doc, triple_double, "\n",
indent_docstring(autodoc, indent), "\n",
indent_docstring(str, indent), indent, triple_double, NIL);

blithely leaks the strings returned by indent_docstring(). Now, again, I'm
not saying it's necessarily a problem (even though I do admit that it's
sometimes difficult to suppress my instinctive attempts to fix them), but
I'd like to have a confirmation that we don't care about them and then stop
using Delete() at all because it's just inconsistent and confusing to use
it in some places but not in similar situations elsewhere.

Of course, there is also the possibility of switching all this code to
more idiomatic C++ and return {auto,unique}_ptr<> instead of raw pointers
everywhere, but I don't personally have time/energy to do it and it seems
unlikely to happen.

But I do feel like we should settle on either trying to avoid leaks at
least in the new code or stop using Delete() at all and just make it a NOP.
It could probably make SWIG run faster, too...

What do you think?
VZ
William S Fulton
2016-05-26 07:01:26 UTC
Permalink
Post by Vadim Zeitlin
Hello,
Currently there is an enormous number of memory leaks in SWIG code as
plenty of DOH objects are never freed. It could be argued that it doesn't
really matter for a program such as SWIG which doesn't run for a long time
and so it doesn't matter if it leaks memory anyhow as all of it will be
reclaimed by the OS on process termination. However the problem is that
there are still _some_ calls to Delete() in SWIG code. And I never know if
I should care about memory being leaked and try to at least call Delete()
correctly in the new code or not.
To make this more concrete, I'm trying merging the latest master into the
doxygen branch. There are plenty of non-trivial conflicts as handling of
docstring indentation in master was completely modified in the meanwhile
(commit fa282b3540c87927fc3562bd0f0863accf6a853f and others), so I have to
change some code manually. My existing code in the branch tries to avoid
Printv(doc, triple_double, "\n",
indent_docstring(autodoc, indent), "\n",
indent_docstring(str, indent), indent, triple_double, NIL);
blithely leaks the strings returned by indent_docstring(). Now, again, I'm
not saying it's necessarily a problem (even though I do admit that it's
sometimes difficult to suppress my instinctive attempts to fix them), but
I'd like to have a confirmation that we don't care about them and then stop
using Delete() at all because it's just inconsistent and confusing to use
it in some places but not in similar situations elsewhere.
Of course, there is also the possibility of switching all this code to
more idiomatic C++ and return {auto,unique}_ptr<> instead of raw pointers
everywhere, but I don't personally have time/energy to do it and it seems
unlikely to happen.
But I do feel like we should settle on either trying to avoid leaks at
least in the new code or stop using Delete() at all and just make it a NOP.
Delete should be called to stop memory explosion. It is well known
that overall there are leaks and that it doesn't actually matter. I
don't think it is too important, but I always try and call Delete if
I'm writing new code. At the end of the day it doesn't really matter
so long as the code is not in some loop that is called multiple times.
I wouldn't go to the trouble of adding in any Delete calls to existing
code unless it is in an area of code that is being modified.

William
Vadim Zeitlin
2016-05-26 13:34:02 UTC
Permalink
On Thu, 26 May 2016 08:01:26 +0100 William S Fulton <***@fultondesigns.co.uk> wrote:

WSF> On 25 May 2016 at 23:17, Vadim Zeitlin <vz-***@zeitlins.org> wrote:
...
WSF> > To make this more concrete, I'm trying merging the latest master into the
WSF> > doxygen branch. There are plenty of non-trivial conflicts as handling of
WSF> > docstring indentation in master was completely modified in the meanwhile
WSF> > (commit fa282b3540c87927fc3562bd0f0863accf6a853f and others), so I have to
WSF> > change some code manually. My existing code in the branch tries to avoid
WSF> > memory leaks but the new code doesn't, at all, e.g. this:
WSF> >
WSF> > Printv(doc, triple_double, "\n",
WSF> > indent_docstring(autodoc, indent), "\n",
WSF> > indent_docstring(str, indent), indent, triple_double, NIL);
WSF> >
WSF> > blithely leaks the strings returned by indent_docstring().
...
WSF> > But I do feel like we should settle on either trying to avoid leaks at
WSF> > least in the new code or stop using Delete() at all and just make it a NOP.
WSF>
WSF> Delete should be called to stop memory explosion. It is well known
WSF> that overall there are leaks and that it doesn't actually matter. I
WSF> don't think it is too important, but I always try and call Delete if
WSF> I'm writing new code.

Well, the problem is that the code above _is_ new, yet it still just leaks
memory. I agree that it (probably?) doesn't really matter, but if it
doesn't, then why do we call Delete() at all anywhere? Is memory explosion
really that big of a problem?

For me the problem is that after the merge the code is glaringly
inconsistent: I have my existing code which uses temporary variables, i.e.
does the equivalent of

String* tmp = indent_docstring(...);
Printv(doc, tmp, NIL);
Delete(tmp);

and code like Printv() above. I'd just like to standardize on one or the
other, but not have both side by side.

Which one would you prefer?
VZ
Phineas Campbell
2016-05-26 15:06:26 UTC
Permalink
Bottom post?

Because that is what I've done

-----Original Message-----
From: Vadim Zeitlin [mailto:vz-***@zeitlins.org]
Sent: 26 May 2016 14:34
To: swig-devel
Subject: Re: [Swig-devel] Should new code call Delete() at all?

On Thu, 26 May 2016 08:01:26 +0100 William S Fulton
<***@fultondesigns.co.uk> wrote:

WSF> On 25 May 2016 at 23:17, Vadim Zeitlin <vz-***@zeitlins.org> wrote:
...
WSF> > To make this more concrete, I'm trying merging the latest master
WSF> > into the doxygen branch. There are plenty of non-trivial
WSF> > conflicts as handling of docstring indentation in master was
WSF> > completely modified in the meanwhile (commit
WSF> > fa282b3540c87927fc3562bd0f0863accf6a853f and others), so I have
WSF> > to change some code manually. My existing code in the branch tries to
avoid memory leaks but the new code doesn't, at all, e.g. this:
WSF> >
WSF> > Printv(doc, triple_double, "\n",
WSF> > indent_docstring(autodoc, indent), "\n",
WSF> > indent_docstring(str, indent), indent,
WSF> > triple_double, NIL);
WSF> >
WSF> > blithely leaks the strings returned by indent_docstring().
...
WSF> > But I do feel like we should settle on either trying to avoid
WSF> > leaks at least in the new code or stop using Delete() at all and just
make it a NOP.
WSF>
WSF> Delete should be called to stop memory explosion. It is well known
WSF> that overall there are leaks and that it doesn't actually matter. I
WSF> don't think it is too important, but I always try and call Delete
WSF> if I'm writing new code.

Well, the problem is that the code above _is_ new, yet it still just leaks
memory. I agree that it (probably?) doesn't really matter, but if it
doesn't, then why do we call Delete() at all anywhere? Is memory explosion
really that big of a problem?

For me the problem is that after the merge the code is glaringly
inconsistent: I have my existing code which uses temporary variables, i.e.
does the equivalent of

String* tmp = indent_docstring(...);
Printv(doc, tmp, NIL);
Delete(tmp);

and code like Printv() above. I'd just like to standardize on one or the
other, but not have both side by side.

Which one would you prefer?
VZ


Is indent_docstring() the only potential source of leaks?

If it is then I'd be willing to run a find at least to count them and
identify how many fixes need to be made

Phineas
'Vadim Zeitlin'
2016-05-26 16:45:17 UTC
Permalink
On Thu, 26 May 2016 16:06:26 +0100 Phineas Campbell <***@phincampbell.com> wrote:

PC> Is indent_docstring() the only potential source of leaks?

No, not at all, it was just a concrete example. There are plenty of
others, but this one is (a) relatively new and (b) very inconsistent with
the use of a similar function in doxygen branch code.

PC> If it is then I'd be willing to run a find at least to count them and
PC> identify how many fixes need to be made

I'm not optimistic about this, IMHO there are too many. I'd be more
interested in making Delete() a NOP and running SWIG with some big input
file to check just how much memory exactly would it consume if it didn't
free it at all. I think it might be not that much... Running it on a random
example in the test suite under valgrind I see that it allocates ~60MB (of
which ~10MB are in use on exit but just 50KB are definitely leaked, much
less than I would have thought). Even if it allocates 10 times this for a
bigger project, 600MB of heap allocation is not that much nowadays. And it
could well make it run faster, because while SWIG memory consumption has
never been a problem for me, its run-time definitely was/is.

Regards,
VZ
William S Fulton
2016-05-27 18:25:43 UTC
Permalink
Post by 'Vadim Zeitlin'
PC> Is indent_docstring() the only potential source of leaks?
No, not at all, it was just a concrete example. There are plenty of
others, but this one is (a) relatively new and (b) very inconsistent with
the use of a similar function in doxygen branch code.
PC> If it is then I'd be willing to run a find at least to count them and
PC> identify how many fixes need to be made
I'm not optimistic about this, IMHO there are too many. I'd be more
interested in making Delete() a NOP and running SWIG with some big input
file to check just how much memory exactly would it consume if it didn't
free it at all. I think it might be not that much... Running it on a random
example in the test suite under valgrind I see that it allocates ~60MB (of
which ~10MB are in use on exit but just 50KB are definitely leaked, much
less than I would have thought). Even if it allocates 10 times this for a
bigger project, 600MB of heap allocation is not that much nowadays. And it
could well make it run faster, because while SWIG memory consumption has
never been a problem for me, its run-time definitely was/is.
Memory explosion is a problem. About 10 years ago Marcelo fixed a lot
of memory leaks because SWIG leaked so much that some wrappers could
not be generated. So making Delete() a NOP is not an option. However,
as it is mostly under control now some leaks are harmless. I'd prefer
to see consistency too in calling indent_docstring() to match whatever
is the norm right now.

William
Vadim Zeitlin
2016-05-27 19:17:13 UTC
Permalink
On Fri, 27 May 2016 19:25:43 +0100 William S Fulton <***@fultondesigns.co.uk> wrote:

WSF> Memory explosion is a problem. About 10 years ago Marcelo fixed a lot
WSF> of memory leaks because SWIG leaked so much that some wrappers could
WSF> not be generated.

The amount of memory in an average desktop machine has probably at least
doubled since then though (without speaking of an almost universal switch
to 64 bit OS with their larger address space).

WSF> So making Delete() a NOP is not an option. However, as it is mostly
WSF> under control now some leaks are harmless. I'd prefer to see
WSF> consistency too in calling indent_docstring() to match whatever is the
WSF> norm right now.

I have no idea what is the norm :-(

What do you think of using smart pointers to make writing the code without
leaking memory less burdensome? E.g.:
---------------------------------- >8 --------------------------------------
// Delete a DOH object on scope exit.
class scoped_dohptr
{
public:
explicit scoped_dohptr(DOH* obj) : obj_(obj) {}
~scoped_dohptr() { Delete(obj_); }

// This is an std::auto_ptr<>-like "destructive" copy ctor which allows to return objects of this type from functions.
scoped_dohptr(scoped_dohptr const& tmp) : obj_(tmp.release()) {}

DOH* get() const { return obj_; }

DOH* release() const /* not really */ {
DOH* obj = obj_;
const_cast<DOH*&>(const_cast<scoped_dohptr*>(this)->obj_) = NULL;
return obj;
}

operator DOH*() const { return obj_; }

private:
scoped_dohptr& operator=(scoped_dohptr const&);

DOH* const obj_;
};
---------------------------------- >8 --------------------------------------

Would you accept a patch adding this class and starting to use it in new
code?
VZ
William S Fulton
2016-05-28 00:12:18 UTC
Permalink
Post by Vadim Zeitlin
WSF> Memory explosion is a problem. About 10 years ago Marcelo fixed a lot
WSF> of memory leaks because SWIG leaked so much that some wrappers could
WSF> not be generated.
The amount of memory in an average desktop machine has probably at least
doubled since then though (without speaking of an almost universal switch
to 64 bit OS with their larger address space).
And the size of code these days seems to have quadrupled!
Post by Vadim Zeitlin
What do you think of using smart pointers to make writing the code without
Would you accept a patch adding this class and starting to use it in new
code?
No, I value consistency more than multiple styles. If we are to move
to smart pointers, let's put in everywhere and get the choice of smart
pointer right (that will probably take some debate). Given the
stretched resources, I think there are more important things to deal
with. Once we have the main patches merged into swig-3.1, it might
then be worth considering some code refactoring.

William
Vadim Zeitlin
2016-05-28 15:14:10 UTC
Permalink
On Sat, 28 May 2016 01:12:18 +0100 William S Fulton <***@fultondesigns.co.uk> wrote:

WSF> On 27 May 2016 at 20:17, Vadim Zeitlin <vz-***@zeitlins.org> wrote:
WSF> > On Fri, 27 May 2016 19:25:43 +0100 William S Fulton <***@fultondesigns.co.uk> wrote:
WSF> >
WSF> > WSF> Memory explosion is a problem. About 10 years ago Marcelo fixed a lot
WSF> > WSF> of memory leaks because SWIG leaked so much that some wrappers could
WSF> > WSF> not be generated.
WSF> >
WSF> > The amount of memory in an average desktop machine has probably at least
WSF> > doubled since then though (without speaking of an almost universal switch
WSF> > to 64 bit OS with their larger address space).
WSF> >
WSF> And the size of code these days seems to have quadrupled!

Well, I finally decided to do my test myself and rebuilt SWIG with
-DDohFree="(void)" and then ran it with "-freeze" option on my own project
and looked at the heap size. It was slightly larger than 2GB, so I guess it
isn't quite practical to stop freeing the memory yet...

WSF> > What do you think of using smart pointers to make writing the code without
WSF> > leaking memory less burdensome? E.g.:
WSF>
WSF> > Would you accept a patch adding this class and starting to use it in new
WSF> > code?
WSF> No, I value consistency more than multiple styles.

I understand and normally agree, i.e. I hadn't proposed doing this before
because I was pretty sure you would want to update the entire code base at
once and I was not ready to propose doing this. But I start to think that
it might be a mistake because it looks like this is just never going to
happen at all if we wait until somebody has the time to do it.

WSF> If we are to move to smart pointers, let's put in everywhere and get
WSF> the choice of smart pointer right (that will probably take some
WSF> debate).

So instead of wasting time on bike shedding it might be better if we just
started using some smart pointer somewhere. It would be trivial to replace
this smart pointer with another one later (because any smart pointer is
greppable). I won't insist, of course, but, due to your long familiarity
with it, you might not realize just how frustrating can it be to work with
SWIG code. IMO making it less so should be given a higher priority.

Regards,
VZ
Clément David
2016-06-02 08:21:19 UTC
Permalink
Hi all,

my 2¢ on this topic, why not using the free Coverity Scan [1] to improve the situation ?

We use it at Scilab to ease external contribution and it works quiet well. The Coverity UI is nice
and can track and display any regression. It is also integrated with github and travis so it should
not be too hard to setup.

[1]: https://scan.coverity.com/

Regards,

--
Clément
Post by Vadim Zeitlin
WSF> >
WSF> > WSF> Memory explosion is a problem. About 10 years ago Marcelo fixed a lot
WSF> > WSF> of memory leaks because SWIG leaked so much that some wrappers could
WSF> > WSF> not be generated.
WSF> >
WSF> >  The amount of memory in an average desktop machine has probably at least
WSF> > doubled since then though (without speaking of an almost universal switch
WSF> > to 64 bit OS with their larger address space).
WSF> >
WSF> And the size of code these days seems to have quadrupled!
 Well, I finally decided to do my test myself and rebuilt SWIG with
-DDohFree="(void)" and then ran it with "-freeze" option on my own project
and looked at the heap size. It was slightly larger than 2GB, so I guess it
isn't quite practical to stop freeing the memory yet...
WSF> >  What do you think of using smart pointers to make writing the code without
WSF> 
WSF> >  Would you accept a patch adding this class and starting to use it in new
WSF> > code?
WSF> No, I value consistency more than multiple styles.
 I understand and normally agree, i.e. I hadn't proposed doing this before
because I was pretty sure you would want to update the entire code base at
once and I was not ready to propose doing this. But I start to think that
it might be a mistake because it looks like this is just never going to
happen at all if we wait until somebody has the time to do it.
WSF> If we are to move to smart pointers, let's put in everywhere and get
WSF> the choice of smart pointer right (that will probably take some
WSF> debate).
 So instead of wasting time on bike shedding it might be better if we just
started using some smart pointer somewhere. It would be trivial to replace
this smart pointer with another one later (because any smart pointer is
greppable). I won't insist, of course, but, due to your long familiarity
with it, you might not realize just how frustrating can it be to work with
SWIG code. IMO making it less so should be given a higher priority.
 Regards,
VZ
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Phineas Campbell
2016-06-02 08:56:24 UTC
Permalink
I've been doing a bit of work memory allocation and have more or less documented all the areas where memory might be being leaked, given the current situation I am not going to give any details until things have quietened down a bit

Phineas


-----Original Message-----
From: Clément David [mailto:***@scilab-enterprises.com]
Sent: 02 June 2016 09:21
To: swig-***@lists.sourceforge.net
Subject: Re: [Swig-devel] Should new code call Delete() at all?

Hi all,

my 2¢ on this topic, why not using the free Coverity Scan [1] to improve the situation ?

We use it at Scilab to ease external contribution and it works quiet well. The Coverity UI is nice and can track and display any regression. It is also integrated with github and travis so it should not be too hard to setup.

[1]: https://scan.coverity.com/

Regards,

--
Clément
Post by Vadim Zeitlin
WSF> >
WSF> > WSF> Memory explosion is a problem. About 10 years ago Marcelo
WSF> > WSF> fixed a lot of memory leaks because SWIG leaked so much
WSF> > WSF> that some wrappers could not be generated.
WSF> >
WSF> > The amount of memory in an average desktop machine has
WSF> >probably at least doubled since then though (without speaking
WSF> >of an almost universal switch to 64 bit OS with their larger address space).
WSF> >
WSF> And the size of code these days seems to have quadrupled!
Well, I finally decided to do my test myself and rebuilt SWIG with
-DDohFree="(void)" and then ran it with "-freeze" option on my own
project and looked at the heap size. It was slightly larger than 2GB,
so I guess it isn't quite practical to stop freeing the memory yet...
WSF> > What do you think of using smart pointers to make writing the
WSF>
WSF> > Would you accept a patch adding this class and starting to use
WSF> >it in new code?
WSF> No, I value consistency more than multiple styles.
I understand and normally agree, i.e. I hadn't proposed doing this
before because I was pretty sure you would want to update the entire
code base at once and I was not ready to propose doing this. But I
start to think that it might be a mistake because it looks like this
is just never going to happen at all if we wait until somebody has the time to do it.
WSF> If we are to move to smart pointers, let's put in everywhere and
WSF> get the choice of smart pointer right (that will probably take
WSF> some debate).
So instead of wasting time on bike shedding it might be better if we
just started using some smart pointer somewhere. It would be trivial
to replace this smart pointer with another one later (because any
smart pointer is greppable). I won't insist, of course, but, due to
your long familiarity with it, you might not realize just how
frustrating can it be to work with SWIG code. IMO making it less so should be given a higher priority.
Regards,
VZ
----------------------------------------------------------------------
-------- What NetFlow Analyzer can do for you? Monitors network
bandwidth and traffic patterns at an interface-level. Reveals which
users, apps, and protocols are consuming the most bandwidth. Provides
multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make
informed decisions using capacity planning reports.
https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
Swig-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/swig-devel
William S Fulton
2016-06-04 12:35:15 UTC
Permalink
Memory leaks are not an issue right now. To summarise:

- Leaking every memory allocation is not acceptable as too much memory
is required for many users.
- It is okay to leak some memory (in fact it is arguably better for
improved performance).
- The current balance seems to be fine. If someone thinks it is not,
then please provide some examples and details of the problem and then
I'll consider patches to reduce memory leakage in targeted areas.

Coverity scan was set up some while ago and is useful for invalid
memory access. I don't see a need for it to report memory leaks. Look
for the coverity-scan branch on Travis at
https://travis-ci.org/swig/swig/branches. I found Coverity a bit
awkward to use as it couldn't be run on Travis on every commit due to
a limitation on the number of times it could be run per week. It also
takes a long time to run and needs quite a bit of manual checking and
logging into another server where users need explicity setting up so
it isn't very public. Maybe the limitations are gone now. I would be
very happy if someone stepped forward and took a new look at Coverity
and made it more automated. Clément, maybe you can give some links
showing how your set up works?

William
Post by Phineas Campbell
I've been doing a bit of work memory allocation and have more or less documented all the areas where memory might be being leaked, given the current situation I am not going to give any details until things have quietened down a bit
Phineas
-----Original Message-----
Sent: 02 June 2016 09:21
Subject: Re: [Swig-devel] Should new code call Delete() at all?
Hi all,
my 2¢ on this topic, why not using the free Coverity Scan [1] to improve the situation ?
We use it at Scilab to ease external contribution and it works quiet well. The Coverity UI is nice and can track and display any regression. It is also integrated with github and travis so it should not be too hard to setup.
[1]: https://scan.coverity.com/
Regards,
--
Clément
Post by Vadim Zeitlin
WSF> >
WSF> > WSF> Memory explosion is a problem. About 10 years ago Marcelo
WSF> > WSF> fixed a lot of memory leaks because SWIG leaked so much
WSF> > WSF> that some wrappers could not be generated.
WSF> >
WSF> > The amount of memory in an average desktop machine has
WSF> >probably at least doubled since then though (without speaking
WSF> >of an almost universal switch to 64 bit OS with their larger address space).
WSF> >
WSF> And the size of code these days seems to have quadrupled!
Well, I finally decided to do my test myself and rebuilt SWIG with
-DDohFree="(void)" and then ran it with "-freeze" option on my own
project and looked at the heap size. It was slightly larger than 2GB,
so I guess it isn't quite practical to stop freeing the memory yet...
WSF> > What do you think of using smart pointers to make writing the
WSF>
WSF> > Would you accept a patch adding this class and starting to use
WSF> >it in new code?
WSF> No, I value consistency more than multiple styles.
I understand and normally agree, i.e. I hadn't proposed doing this
before because I was pretty sure you would want to update the entire
code base at once and I was not ready to propose doing this. But I
start to think that it might be a mistake because it looks like this
is just never going to happen at all if we wait until somebody has the time to do it.
WSF> If we are to move to smart pointers, let's put in everywhere and
WSF> get the choice of smart pointer right (that will probably take
WSF> some debate).
So instead of wasting time on bike shedding it might be better if we
just started using some smart pointer somewhere. It would be trivial
to replace this smart pointer with another one later (because any
smart pointer is greppable). I won't insist, of course, but, due to
your long familiarity with it, you might not realize just how
frustrating can it be to work with SWIG code. IMO making it less so should be given a higher priority.
Regards,
VZ
----------------------------------------------------------------------
-------- What NetFlow Analyzer can do for you? Monitors network
bandwidth and traffic patterns at an interface-level. Reveals which
users, apps, and protocols are consuming the most bandwidth. Provides
multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make
informed decisions using capacity planning reports.
https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Clément David
2016-06-06 08:10:52 UTC
Permalink
Hi William, hi all,
After a few reading, the travis_ci / github default integration seems to be quiet manual [1]. As you
have already set it up, it requires a specific branch and you have to merge to trigger a coverity
scan.

At Scilab, we have our own Jenkins setup [2] that trigger a coverity scan each working day
(accordingly to the submission limits [3]). It is launched on the master branch and requires no user
intervention. IMHO it is simpler to use than "merge to trigger a build" : we validate every commits
that claim to fix coverity issues without adding regressions. We also do not use the coverity UI to
assign / edit issues.

AFAIK this kind of setup might be reproduced using travis CI "Cron jobs" but I have no knowledge of
it.

[1]: https://scan.coverity.com/travis_ci
[2]: http://build.scilab.org/job/scilab-build-coverity-centos
[3]: https://scan.coverity.com/faq#frequency
[3]: https://docs.travis-ci.com/user/cron-jobs/

--
Clément
Post by William S Fulton
- Leaking every memory allocation is not acceptable as too much memory
is required for many users.
- It is okay to leak some memory (in fact it is arguably better for
improved performance).
- The current balance seems to be fine. If someone thinks it is not,
then please provide some examples and details of the problem and then
I'll consider patches to reduce memory leakage in targeted areas.
Coverity scan was set up some while ago and is useful for invalid
memory access. I don't see a need for it to report memory leaks. Look
for the coverity-scan branch on Travis at
https://travis-ci.org/swig/swig/branches. I found Coverity a bit
awkward to use as it couldn't be run on Travis on every commit due to
a limitation on the number of times it could be run per week. It also
takes a long time to run and needs quite a bit of manual checking and
logging into another server where users need explicity setting up so
it isn't very public. Maybe the limitations are gone now. I would be
very happy if someone stepped forward and took a new look at Coverity
and made it more automated. Clément, maybe you can give some links
showing how your set up works?
William
Post by Phineas Campbell
I've been doing a bit of work memory allocation and have more or less documented all the areas
where memory might be being leaked, given the current situation I am not going to give any
details until things have quietened down a bit
Phineas
-----Original Message-----
Sent: 02 June 2016 09:21
Subject: Re: [Swig-devel] Should new code call Delete() at all?
Hi all,
my 2¢ on this topic, why not using the free Coverity Scan [1] to improve the situation ?
We use it at Scilab to ease external contribution and it works quiet well. The Coverity UI is
nice and can track and display any regression. It is also integrated with github and travis so
it should not be too hard to setup.
[1]: https://scan.coverity.com/
Regards,
--
Clément
Post by Vadim Zeitlin
WSF> >
WSF> > WSF> Memory explosion is a problem. About 10 years ago Marcelo
WSF> > WSF> fixed a lot of memory leaks because SWIG leaked so much
WSF> > WSF> that some wrappers could not be generated.
WSF> >
WSF> >  The amount of memory in an average desktop machine has
WSF> >probably at least  doubled since then though (without speaking
WSF> >of an almost universal switch  to 64 bit OS with their larger address space).
WSF> >
WSF> And the size of code these days seems to have quadrupled!
 Well, I finally decided to do my test myself and rebuilt SWIG with
-DDohFree="(void)" and then ran it with "-freeze" option on my own
project and looked at the heap size. It was slightly larger than 2GB,
so I guess it isn't quite practical to stop freeing the memory yet...
WSF> >  What do you think of using smart pointers to make writing the
WSF>
WSF> >  Would you accept a patch adding this class and starting to use
WSF> >it in new  code?
WSF> No, I value consistency more than multiple styles.
 I understand and normally agree, i.e. I hadn't proposed doing this
before because I was pretty sure you would want to update the entire
code base at once and I was not ready to propose doing this. But I
start to think that it might be a mistake because it looks like this
is just never going to happen at all if we wait until somebody has the time to do it.
WSF> If we are to move to smart pointers, let's put in everywhere and
WSF> get the choice of smart pointer right (that will probably take
WSF> some debate).
 So instead of wasting time on bike shedding it might be better if we
just started using some smart pointer somewhere. It would be trivial
to replace this smart pointer with another one later (because any
smart pointer is greppable). I won't insist, of course, but, due to
your long familiarity with it, you might not realize just how
frustrating can it be to work with SWIG code. IMO making it less so should be given a higher
priority.
 Regards,
VZ
----------------------------------------------------------------------
-------- What NetFlow Analyzer can do for you? Monitors network
bandwidth and traffic patterns at an interface-level. Reveals which
users, apps, and protocols are consuming the most bandwidth. Provides
multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make
informed decisions using capacity planning reports.
https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an
interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth.
Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed
decisions using capacity planning reports. https://ad.doubleclick.net/ddm/clk/305295220;13265958
2;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Swig-devel mailing list
https://lists.sourceforge.net/lists/listinfo/swig-devel
Loading...