[Salix-main] Packaging tools updates

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Salix-main] Packaging tools updates

George Vlahavas
Hi!

I have updated some of the packaging tools we're using with new features.
Some are somewhat major, some minor. Here's the list:

slkbuild
========

* If the arch is not set in the SLKBUILD file and is automatically set, it
   is now available earlier and can be used in source(), without having to
   explicitly set it from the command line.
* CFLAGS/CXXFLAGS for 32-bit is now set to arch=i586, instead of i486.
   This is following the respective change in slackware-current for the
   next release.
* The package extension can be set by setting a PKGEXT environment
   variable appropriately (example: PKGEXT=tgz). This will be overriden if
   the package extension is set in the SLKBUILD file.
* Add support for brotli packages (.tbr). Of course you can use the
   PKGEXT variable with this. More on brotli later...
* Add an internal makepkg function. If the slackware pkgtools is not
   present, this will be used instead. Can also be set by using a MAKEPKG
   environment variable (MAKEPKG=makepkg_internal). The MAKEPKG variable
   can also point to any other makepkg alternative, if you find any. This
   internal makepkg supports .tbr packages. It is copied over from a script
   by ruario.

You can find packages here:

http://people.salixos.org/gapan/slkbuild/

spkg
====

* Add support for brotli (.tbr) packages

You can find packages here:

http://people.salixos.org/gapan/spkg/

depfinder
=========

* Add support for brotli (.tbr) packages

You can find packages here:

http://people.salixos.org/gapan/depfinder/

brotli
======

Brotli is a (very) new compression algorithm from google. You can find the
code and a brief description here:
https://github.com/google/brotli

While Slackware is traditionally slow at adopting new things like this and
I'm not sure if they ever adopt it anyway, I don't think there is any harm
in us having our tools ready for it.

I have made some testing and while I don't think we are likely to replace
xz as the default compression algorithm, it certainly is interesting and
can have its uses. While it is not better in absolute terms with respect
to compression ratio than xz, it is close, when it uses the higher
settings. But, in all cases it is blazing fast. Compression speeds are
very fast for anything other than the highest compression levels and
decompression speeds are generally faster than anything else. It is even
faster than decompressing gzip, at any compression level, while at the
same time having better compression ratios.

There is a discussion over at the LQ forums that I participated in further
details:
http://www.linuxquestions.org/questions/slackware-14/google-brotli-what-you-really-want-to-know-%3B-4175554367/

The thread includes links that point to benchmarks and commentary about
brotli. Specifically the speed tests that I have run with respect to
using it for packaging are in this post:
http://www.linuxquestions.org/questions/slackware-14/google-brotli-what-you-really-want-to-know-%3B-4175554367/#post5437396

I'm thinking that this might be a good replacement for the default package
type to use for building local packages with slapt-src/sourcery. Anything
from quality level 1 to 5 should do for that, as sizes are comparable to
the best gzip compression levels and compression/decompression times are
significantly shorter. All this is not for 14.1 of course, rather for our
next release, when it comes.

You can find packages for the latest official release of brotli in the
link below. This needs to be installed if you want to use any of the new
brotli features of all previous tools.

http://people.salixos.org/gapan/brotli/



So, you can use the new slkbuild package to create .tbr packages with
something like:

PKGEXT=tbr MAKEPKG=makepkg_internal fakeroot slkbuild -X

(or remove the pkgtools package so that you don't have to set the MAKEPKG
variable) and then use the updated depfinder package to detect
dependencies and the updated spkg package to install/update/remove it.

It would be nice if someone else tested these as well. Thoughts and
opinions are welcome.

Have fun!

George

------------------------------------------------------------------------------
_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Salix-main] Packaging tools updates

Richard Lapointe
hi,

Tried building a package several times using various combinations of the new environments with and without pkgtools installed.   In all case the packed built or didn't build as expected based on the combination tested.  In the last case that I  tested, where I had pkgtools installed, and PKGEXT set to tbr but i did not set MAKEPKG to makepkg_internal the package did not build as expected, but I did get some messages could be somewhat confusing. Here's  the output of the last several lines:

ERROR:  Package extension .tbr is not supported.
md5sum: ekiga-4.0.1-x86_64-1rl.tbr: No such file or directory
Package has been built.
Cleaning pkg and src directories

So it does imply that the package was built. Also, is there anyway to detect this before compiling?

Thanks

Rich (laprjns)

On 10/30/2015 01:58 PM, George Vlahavas wrote:
Hi!

I have updated some of the packaging tools we're using with new features. 
Some are somewhat major, some minor. Here's the list:

slkbuild
========

* If the arch is not set in the SLKBUILD file and is automatically set, it
   is now available earlier and can be used in source(), without having to
   explicitly set it from the command line.
* CFLAGS/CXXFLAGS for 32-bit is now set to arch=i586, instead of i486.
   This is following the respective change in slackware-current for the
   next release.
* The package extension can be set by setting a PKGEXT environment
   variable appropriately (example: PKGEXT=tgz). This will be overriden if
   the package extension is set in the SLKBUILD file.
* Add support for brotli packages (.tbr). Of course you can use the
   PKGEXT variable with this. More on brotli later...
* Add an internal makepkg function. If the slackware pkgtools is not
   present, this will be used instead. Can also be set by using a MAKEPKG
   environment variable (MAKEPKG=makepkg_internal). The MAKEPKG variable
   can also point to any other makepkg alternative, if you find any. This
   internal makepkg supports .tbr packages. It is copied over from a script
   by ruario.

You can find packages here:

http://people.salixos.org/gapan/slkbuild/

spkg
====

* Add support for brotli (.tbr) packages

You can find packages here:

http://people.salixos.org/gapan/spkg/

depfinder
=========

* Add support for brotli (.tbr) packages

You can find packages here:

http://people.salixos.org/gapan/depfinder/

brotli
======

Brotli is a (very) new compression algorithm from google. You can find the 
code and a brief description here:
https://github.com/google/brotli

While Slackware is traditionally slow at adopting new things like this and 
I'm not sure if they ever adopt it anyway, I don't think there is any harm 
in us having our tools ready for it.

I have made some testing and while I don't think we are likely to replace 
xz as the default compression algorithm, it certainly is interesting and 
can have its uses. While it is not better in absolute terms with respect 
to compression ratio than xz, it is close, when it uses the higher 
settings. But, in all cases it is blazing fast. Compression speeds are 
very fast for anything other than the highest compression levels and 
decompression speeds are generally faster than anything else. It is even 
faster than decompressing gzip, at any compression level, while at the 
same time having better compression ratios.

There is a discussion over at the LQ forums that I participated in further 
details:
http://www.linuxquestions.org/questions/slackware-14/google-brotli-what-you-really-want-to-know-%3B-4175554367/

The thread includes links that point to benchmarks and commentary about 
brotli. Specifically the speed tests that I have run with respect to 
using it for packaging are in this post: 
http://www.linuxquestions.org/questions/slackware-14/google-brotli-what-you-really-want-to-know-%3B-4175554367/#post5437396

I'm thinking that this might be a good replacement for the default package 
type to use for building local packages with slapt-src/sourcery. Anything 
from quality level 1 to 5 should do for that, as sizes are comparable to 
the best gzip compression levels and compression/decompression times are 
significantly shorter. All this is not for 14.1 of course, rather for our 
next release, when it comes.

You can find packages for the latest official release of brotli in the 
link below. This needs to be installed if you want to use any of the new 
brotli features of all previous tools.

http://people.salixos.org/gapan/brotli/



So, you can use the new slkbuild package to create .tbr packages with 
something like:

PKGEXT=tbr MAKEPKG=makepkg_internal fakeroot slkbuild -X

(or remove the pkgtools package so that you don't have to set the MAKEPKG 
variable) and then use the updated depfinder package to detect 
dependencies and the updated spkg package to install/update/remove it.

It would be nice if someone else tested these as well. Thoughts and 
opinions are welcome.

Have fun!

George

------------------------------------------------------------------------------
_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main


------------------------------------------------------------------------------

_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Salix-main] Packaging tools updates

George Vlahavas
Hi Rich,

On Sat, 31 Oct 2015, Richard Lapointe wrote:

> Tried building a package several times using various combinations of the new
> environments with and without pkgtools installed.   In all case the packed
> built or didn't build as expected based on the combination tested.  In the
> last case that I  tested, where I had pkgtools installed, and PKGEXT set to
> tbr but i did not set MAKEPKG to makepkg_internal the package did not build
> as expected, but I did get some messages could be somewhat confusing. Here's
> the output of the last several lines:
>
> /ERROR:  Package extension .tbr is not supported.//
> //md5sum: ekiga-4.0.1-x86_64-1rl.tbr: No such file or directory//
> //Package has been built.//
> //Cleaning pkg and src directories/
>
> So it does imply that the package was built.

Correct. Everything that was needed was built and put in the pkg
directory. However, since the slackware makepkg does not support tbr at
this point, it was impossible to create a tbr package as requested.
Furthermore, you used the -X switch and the pkg directory was deleted. If
you had only used the -x switch, the pkg directory would still be there
and you could have cd'd into it and ran makepkg manually.

> Also, is there anyway to detect this before compiling?

Sure! Now if PKGEXT=tbr, it should use the internal makepkg, regardless of
what the MAKEPKG variable says or if pkgtools is installed. New package is
in the same place:

http://people.salixos.org/gapan/slkbuild/

(now that I think more about it, this doesn't cover the case where 'tbr'
is set inside the SLKBUILD file options(). I'll see what I'll do about
that too.)

Thanks for the testing Rich!

George

------------------------------------------------------------------------------
_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Salix-main] Packaging tools updates

George Vlahavas
In reply to this post by Richard Lapointe
OK, so now if 'tbr' is set inside the SLKBUILD in options(), it overrides
any other setting and uses the internal makepkg too. The updated packages
(versioned 20151031b) are at the same location:

http://people.salixos.org/gapan/slkbuild/

------------------------------------------------------------------------------
_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Salix-main] Packaging tools updates

Richard Lapointe
I think you didn't get the last update into the package.  When I tried
to build a package with PKGEXT=tbr set in the SLKBUILD it still failed
saying that .tbr is not supported

Looking at the slkbuild script in the 20151031b package it say:

     slkbuildver=1.0beta1

Rich

On 10/31/2015 02:37 PM, George Vlahavas wrote:
> OK, so now if 'tbr' is set inside the SLKBUILD in options(), it
> overrides any other setting and uses the internal makepkg too. The
> updated packages (versioned 20151031b) are at the same location:
>
> http://people.salixos.org/gapan/slkbuild/


------------------------------------------------------------------------------
_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Salix-main] Packaging tools updates

George Vlahavas

On Sun, 1 Nov 2015, Richard Lapointe wrote:

> I think you didn't get the last update into the package.  When I tried to
> build a package with PKGEXT=tbr set in the SLKBUILD it still failed saying
> that .tbr is not supported

Ah, I think you got that wrong. If you're going to set the extension in
the SLKBUILD file, you need to do it in options(). Like this:

options=('tbr')

just like with all other extensions.

------------------------------------------------------------------------------
_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Salix-main] Packaging tools updates

Richard Lapointe
Yup, your right, I'm wrong (again).  It's working now.

Rich

On 11/01/2015 05:48 AM, George Vlahavas wrote:

>
> On Sun, 1 Nov 2015, Richard Lapointe wrote:
>
>> I think you didn't get the last update into the package.  When I
>> tried to build a package with PKGEXT=tbr set in the SLKBUILD it still
>> failed saying that .tbr is not supported
>
> Ah, I think you got that wrong. If you're going to set the extension
> in the SLKBUILD file, you need to do it in options(). Like this:
>
> options=('tbr')
>
> just like with all other extensions.


------------------------------------------------------------------------------
_______________________________________________
Salix-main mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/salix-main
Loading...