From rabin at rabin.io Tue Aug 2 18:40:05 2016 From: rabin at rabin.io (Rabin Yasharzadehe) Date: Tue, 2 Aug 2016 18:40:05 +0300 Subject: Raspberry PI - analog sensors with Arduino In-Reply-To: <20160731123751.01960785@shlomo1.solomon> References: <20160731085048.0be080e9@shlomo1.solomon> <20160731123751.01960785@shlomo1.solomon> Message-ID: ?? ?????? ??? ???? ????? ??? ????? ??'???? ??? ???? ??????? ??????, ???? ??????? ???? ?? ?? ?????? ???? ?? ???????? ?? ?-CHIP ??? ?????? ???? ??????! ??? ?? ?? ????? ?? ?? ????? ?? ???????? ??????? ???? ??? RS232, ??? ?????? ????? ????. ?? ??? ???? ADC ???? ????? ?????? ?? AS-IS ??? ???? ????? ?? ?-DATASHEET ??? ??? ????? ???? ????? ???? ???? ?????. ?????? ???? ???? ???? ?????? ????? ????? ?? ?????? ??? ???? ??????. -- Rabin On 31 July 2016 at 12:37, Shlomo Solomon wrote: > Thanks to Kobi and Jason. > > In the meantime, I've done more research, and I see that there are > several ADC (analog digital converter) chips available to help the PI > use analog input. > Since my main purpose is to learn to use and program the GPIO pins on > the PI, I guess that would be a good solution - especially since my C++ > skills are REALLY rusty and I'm much more comfortable with Python. > > Again - thanks for your replies > > > On Sun, 31 Jul 2016 09:25:31 +0300 > Kobi Zamir wrote: > > > Hi, I also think like Json. > > > > For the first time you play with an arduino, use a board that has: > > > > 1. a usb connector for programming and i/o > > 2. pre-soldered connectors > > 3. use standard arduino connector arrangement (like in the arduino > > uno) > > > > For example: > > > http://www.ebay.com/itm/UNO-R3-ATmega328P-Development-Board-for-Arduino-Compatible-Free-USB-Cable-/191617917471 > > > > > > > > On Sun, Jul 31, 2016 at 9:03 AM, Jason Friedman > > wrote: > > > > > > > > > > >> I have absolutely no knowledge about the Arduino, but I've seen > > >> clones advertised on e-bay for less than $2 - link below. > > >> > > >> Can anyone tell me if this as actually a working solution and if > > >> the low price is actually possible? > > >> > > >> > > >> > > >> > http://www.ebay.com/itm/1PCS-Pro-Mini-atmega328-5V-16M-Replace-ATmega128-Arduino-Compatible-Nano-/152160908037?hash=item236d7f3305:g:MMcAAOSw2GlXLD~U > > >> > > >> > > > Arduino is probably the easiest / cheapest way to access analog > > > sensors. While quality of these do vary, I have had good experience > > > with these very cheap units. Note that you need to program these > > > devices via a USB cable and an FTDI programmer (you can also find > > > one on ebay for a few dollars). I would recommend for someone new > > > to Arduino that you get an Arduino uno / sparkfun redboard (or a > > > chinese clone of one of these) - they have the programmer built > > > into the board (so you just plug it via USB to your computer). They > > > also have headers soldered onto the board already, so you can > > > connect sensors, LEDs, etc without soldering, which is good for > > > getting started. > > > > > > Jason > > > > > > > > > > > >> -- > > >> Shlomo Solomon > > >> http://the-solomons.net > > >> Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 > > >> > > >> > > >> _______________________________________________ > > >> Linux-il mailing list > > >> Linux-il at cs.huji.ac.il > > >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > >> > > > > > > > > > > > > -- > > > Jason Friedman, PhD > > > Senior Lecturer > > > Department of Physical Therapy > > > Tel Aviv University > > > email: write.to.jason at gmail.com > > > web: http://curiousjason.com > > > > > > _______________________________________________ > > > Linux-il mailing list > > > Linux-il at cs.huji.ac.il > > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > > > > > > > > > -- > Shlomo Solomon > http://the-solomons.net > Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shlomo.solomon at gmail.com Tue Aug 2 19:14:04 2016 From: shlomo.solomon at gmail.com (Shlomo Solomon) Date: Tue, 2 Aug 2016 19:14:04 +0300 Subject: Raspberry PI - analog sensors with Arduino In-Reply-To: References: <20160731085048.0be080e9@shlomo1.solomon> <20160731123751.01960785@shlomo1.solomon> Message-ID: <20160802191404.0db8c70a@shlomo1.solomon> Actually, since my purpose is to experiment and learn , **getting my hands dirty** with datasheets, etc is OK. By what you write, I assume you have some experience or knowledge, so I'll ask what I really should have asked in my original post. There are many sensor kits on e-bay, but although some of them mention Raspberry PI, it seems that they are not "really" meant for the PI, but for Arduino. Assuming, I'm willing to **dirty my hands**, do you think they should work on a PI? I'm including a link to a typical kit. http://www.ebay.com/itm/322176545722?_trksid=p2055119.m1438.l2649&ssPageName=STRK%3AMEBIDX%3AIT On Tue, 2 Aug 2016 18:40:05 +0300 Rabin Yasharzadehe wrote: > ?? ?????? ??? ???? ????? ??? ????? ??'???? ??? ???? ??????? ??????, > ???? ??????? ???? ?? ?? ?????? ???? ?? ???????? ?? ?-CHIP ??? ?????? > ???? ??????! > ??? ?? ?? ????? ?? ?? ????? ?? ???????? ??????? ???? ??? RS232, ??? > ?????? ????? ????. > > ?? ??? ???? ADC ???? ????? ?????? ?? AS-IS ??? ???? ????? ?? > ?-DATASHEET ??? ??? ????? ???? ????? ???? ???? ?????. > ?????? ???? ???? ???? ?????? ????? ????? ?? ?????? ??? ???? ??????. > > -- > Rabin > > On 31 July 2016 at 12:37, Shlomo Solomon > wrote: > > > Thanks to Kobi and Jason. > > > > In the meantime, I've done more research, and I see that there are > > several ADC (analog digital converter) chips available to help the > > PI use analog input. > > Since my main purpose is to learn to use and program the GPIO pins > > on the PI, I guess that would be a good solution - especially since > > my C++ skills are REALLY rusty and I'm much more comfortable with > > Python. > > > > Again - thanks for your replies > > > > > > On Sun, 31 Jul 2016 09:25:31 +0300 > > Kobi Zamir wrote: > > > > > Hi, I also think like Json. > > > > > > For the first time you play with an arduino, use a board that has: > > > > > > 1. a usb connector for programming and i/o > > > 2. pre-soldered connectors > > > 3. use standard arduino connector arrangement (like in the arduino > > > uno) > > > > > > For example: > > > > > http://www.ebay.com/itm/UNO-R3-ATmega328P-Development-Board-for-Arduino-Compatible-Free-USB-Cable-/191617917471 > > > > > > > > > > > > On Sun, Jul 31, 2016 at 9:03 AM, Jason Friedman > > > wrote: > > > > > > > > > > > > > > >> I have absolutely no knowledge about the Arduino, but I've seen > > > >> clones advertised on e-bay for less than $2 - link below. > > > >> > > > >> Can anyone tell me if this as actually a working solution and > > > >> if the low price is actually possible? > > > >> > > > >> > > > >> > > > >> > > http://www.ebay.com/itm/1PCS-Pro-Mini-atmega328-5V-16M-Replace-ATmega128-Arduino-Compatible-Nano-/152160908037?hash=item236d7f3305:g:MMcAAOSw2GlXLD~U > > > >> > > > >> > > > > Arduino is probably the easiest / cheapest way to access analog > > > > sensors. While quality of these do vary, I have had good > > > > experience with these very cheap units. Note that you need to > > > > program these devices via a USB cable and an FTDI programmer > > > > (you can also find one on ebay for a few dollars). I would > > > > recommend for someone new to Arduino that you get an Arduino > > > > uno / sparkfun redboard (or a chinese clone of one of these) - > > > > they have the programmer built into the board (so you just plug > > > > it via USB to your computer). They also have headers soldered > > > > onto the board already, so you can connect sensors, LEDs, etc > > > > without soldering, which is good for getting started. > > > > > > > > Jason > > > > > > > > > > > > > > > >> -- > > > >> Shlomo Solomon > > > >> http://the-solomons.net > > > >> Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 > > > >> > > > >> > > > >> _______________________________________________ > > > >> Linux-il mailing list > > > >> Linux-il at cs.huji.ac.il > > > >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > > >> > > > > > > > > > > > > > > > > -- > > > > Jason Friedman, PhD > > > > Senior Lecturer > > > > Department of Physical Therapy > > > > Tel Aviv University > > > > email: write.to.jason at gmail.com > > > > web: http://curiousjason.com > > > > > > > > _______________________________________________ > > > > Linux-il mailing list > > > > Linux-il at cs.huji.ac.il > > > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > > > > > > > > > > > > > > > -- > > Shlomo Solomon > > http://the-solomons.net > > Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 > > > > > > _______________________________________________ > > Linux-il mailing list > > Linux-il at cs.huji.ac.il > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -- Shlomo Solomon http://the-solomons.net Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 From elazarl at gmail.com Tue Aug 2 20:50:06 2016 From: elazarl at gmail.com (Elazar Leibovich) Date: Tue, 2 Aug 2016 20:50:06 +0300 Subject: Gradual installation of debian packages Message-ID: Hi, I'm having a few (say, a few tens) Debian machines, with a local repository defined. In the local repository I have some home made packages I'm building and pushing to the local repository. When I'm upgrading my package, I want to be sure the update wouldn't cause a problem. So I wish to install them on a few percentage of the machines, wait for complaints. If complaints arrive - roll back. Otherwise keep upgrading the whole machines. I'll appreciate your advice and experience of similar situation, I'll appreciate if someone who had actual real life experience with this situation would mention it in the comments. Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: From rabin at rabin.io Tue Aug 2 21:07:24 2016 From: rabin at rabin.io (Rabin Yasharzadehe) Date: Tue, 2 Aug 2016 21:07:24 +0300 Subject: Raspberry PI - analog sensors with Arduino In-Reply-To: <20160802191404.0db8c70a@shlomo1.solomon> References: <20160731085048.0be080e9@shlomo1.solomon> <20160731123751.01960785@shlomo1.solomon> <20160802191404.0db8c70a@shlomo1.solomon> Message-ID: ??? ???? ???? ?? ????? ???? (?? ?? ????? ?? ????????), ??? ??? ???? ???? ??????? ????? ????? ??????? ???????? ?? ????? PCB ??? ?? ????? ??????? ???? ???? ???? ?? ?"?????" ???? ?????? ?????? ??? ??? ?? ???? ???? ??-BOARD ?? ?-RPI ??????. ??? ?? ?? ??-RPI ?? ???? ??? ?? ???? ??? ?????? ?? ?????? , ?? ?? ???? ?????? ?? ??? ????? ?????? ??????? ??????? ???? ??? (?????? , ?????? ??"?) ???? ?? ????? ?? ?? ??????? ?? ?-RPI (????? ???? ????? ???? ??? ????? ??? ?? ???? ???\?????????) -- ????? ???? ?? ?????? ????? , ?????? ?? ?-GPIO ?? ?-RPI ??? ???? ???? ???? ?? ????? ?? 5V ?-3.3, ??? ???? ?????? ?? ???? ?????? ???? ???? ?? ????? ??? ????. ??? ??? ????? ???? ????? ???? ????? ????? ?? DATASHEET ????? ???? ?? ?? ??????? ??? ?? ?????. -- Rabin On 2 August 2016 at 19:14, Shlomo Solomon wrote: > Actually, since my purpose is to experiment and learn , **getting my > hands dirty** with datasheets, etc is OK. > > By what you write, I assume you have some experience or knowledge, so > I'll ask what I really should have asked in my original post. There are > many sensor kits on e-bay, but although some of them mention Raspberry > PI, it seems that they are not "really" meant for the PI, but for > Arduino. > > Assuming, I'm willing to **dirty my hands**, do you think they should > work on a PI? > > I'm including a link to a typical kit. > > > http://www.ebay.com/itm/322176545722?_trksid=p2055119.m1438.l2649&ssPageName=STRK%3AMEBIDX%3AIT > > > > > On Tue, 2 Aug 2016 18:40:05 +0300 > Rabin Yasharzadehe wrote: > > > ?? ?????? ??? ???? ????? ??? ????? ??'???? ??? ???? ??????? ??????, > > ???? ??????? ???? ?? ?? ?????? ???? ?? ???????? ?? ?-CHIP ??? ?????? > > ???? ??????! > > ??? ?? ?? ????? ?? ?? ????? ?? ???????? ??????? ???? ??? RS232, ??? > > ?????? ????? ????. > > > > ?? ??? ???? ADC ???? ????? ?????? ?? AS-IS ??? ???? ????? ?? > > ?-DATASHEET ??? ??? ????? ???? ????? ???? ???? ?????. > > ?????? ???? ???? ???? ?????? ????? ????? ?? ?????? ??? ???? ??????. > > > > -- > > Rabin > > > > On 31 July 2016 at 12:37, Shlomo Solomon > > wrote: > > > > > Thanks to Kobi and Jason. > > > > > > In the meantime, I've done more research, and I see that there are > > > several ADC (analog digital converter) chips available to help the > > > PI use analog input. > > > Since my main purpose is to learn to use and program the GPIO pins > > > on the PI, I guess that would be a good solution - especially since > > > my C++ skills are REALLY rusty and I'm much more comfortable with > > > Python. > > > > > > Again - thanks for your replies > > > > > > > > > On Sun, 31 Jul 2016 09:25:31 +0300 > > > Kobi Zamir wrote: > > > > > > > Hi, I also think like Json. > > > > > > > > For the first time you play with an arduino, use a board that has: > > > > > > > > 1. a usb connector for programming and i/o > > > > 2. pre-soldered connectors > > > > 3. use standard arduino connector arrangement (like in the arduino > > > > uno) > > > > > > > > For example: > > > > > > > > http://www.ebay.com/itm/UNO-R3-ATmega328P-Development-Board-for-Arduino-Compatible-Free-USB-Cable-/191617917471 > > > > > > > > > > > > > > > > On Sun, Jul 31, 2016 at 9:03 AM, Jason Friedman > > > > wrote: > > > > > > > > > > > > > > > > > > >> I have absolutely no knowledge about the Arduino, but I've seen > > > > >> clones advertised on e-bay for less than $2 - link below. > > > > >> > > > > >> Can anyone tell me if this as actually a working solution and > > > > >> if the low price is actually possible? > > > > >> > > > > >> > > > > >> > > > > >> > > > > http://www.ebay.com/itm/1PCS-Pro-Mini-atmega328-5V-16M-Replace-ATmega128-Arduino-Compatible-Nano-/152160908037?hash=item236d7f3305:g:MMcAAOSw2GlXLD~U > > > > >> > > > > >> > > > > > Arduino is probably the easiest / cheapest way to access analog > > > > > sensors. While quality of these do vary, I have had good > > > > > experience with these very cheap units. Note that you need to > > > > > program these devices via a USB cable and an FTDI programmer > > > > > (you can also find one on ebay for a few dollars). I would > > > > > recommend for someone new to Arduino that you get an Arduino > > > > > uno / sparkfun redboard (or a chinese clone of one of these) - > > > > > they have the programmer built into the board (so you just plug > > > > > it via USB to your computer). They also have headers soldered > > > > > onto the board already, so you can connect sensors, LEDs, etc > > > > > without soldering, which is good for getting started. > > > > > > > > > > Jason > > > > > > > > > > > > > > > > > > > >> -- > > > > >> Shlomo Solomon > > > > >> http://the-solomons.net > > > > >> Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 > > > > >> > > > > >> > > > > >> _______________________________________________ > > > > >> Linux-il mailing list > > > > >> Linux-il at cs.huji.ac.il > > > > >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > Jason Friedman, PhD > > > > > Senior Lecturer > > > > > Department of Physical Therapy > > > > > Tel Aviv University > > > > > email: write.to.jason at gmail.com > > > > > web: http://curiousjason.com > > > > > > > > > > _______________________________________________ > > > > > Linux-il mailing list > > > > > Linux-il at cs.huji.ac.il > > > > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > > > > > > > > > > > > > > > > > > > > > -- > > > Shlomo Solomon > > > http://the-solomons.net > > > Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 > > > > > > > > > _______________________________________________ > > > Linux-il mailing list > > > Linux-il at cs.huji.ac.il > > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > > > > > > -- > Shlomo Solomon > http://the-solomons.net > Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shlomo.solomon at gmail.com Tue Aug 2 21:14:07 2016 From: shlomo.solomon at gmail.com (Shlomo Solomon) Date: Tue, 2 Aug 2016 21:14:07 +0300 Subject: Raspberry PI - analog sensors with Arduino In-Reply-To: References: <20160731085048.0be080e9@shlomo1.solomon> <20160731123751.01960785@shlomo1.solomon> <20160802191404.0db8c70a@shlomo1.solomon> Message-ID: <20160802211407.46b89b9d@shlomo1.solomon> Thanks - I already ordered a breadboard (similar to the picture you sent) and also a breadboard power supply. On Tue, 2 Aug 2016 21:07:24 +0300 Rabin Yasharzadehe wrote: > ??? ???? ???? ?? ????? ???? (?? ?? ????? ?? ????????), ??? ??? ???? > ???? ??????? ????? ????? ??????? ???????? ?? ????? PCB ??? ?? ????? > ??????? ???? ???? ???? ?? ?"?????" ???? ?????? ?????? > > ??? ??? ?? ???? ???? ??-BOARD ?? ?-RPI ??????. > > ??? ?? ?? ??-RPI ?? ???? ??? ?? ???? ??? ?????? ?? ?????? , ?? ?? ???? > ?????? ?? ??? ????? ?????? ??????? ??????? ???? ??? (?????? , ?????? > ??"?) ???? ?? ????? ?? ?? ??????? ?? ?-RPI (????? ???? ????? ???? ??? > ????? ??? ?? ???? ???\?????????) > -- Shlomo Solomon http://the-solomons.net Sent by Claws Mail 3.11.1 - KDE 4.14.5 - LINUX Mageia 5 From nad.oby at gmail.com Wed Aug 3 09:30:47 2016 From: nad.oby at gmail.com (Evgeniy Ginzburg) Date: Wed, 3 Aug 2016 09:30:47 +0300 Subject: Gradual installation of debian packages In-Reply-To: References: Message-ID: Hello. I'm assuming that you have paswordless ssh to the servers in question as root. Also I assume that you don't use central management/deployment software (ansible/puppet/chef) In similar cases I usully use parallel-ssh (gnu-parallel is another alternative). First stage install the package manually on one server to see that configuration is OK, daemons restart, etc... If this stage is ok second step will be creating list of servers for "complain" list and install package on them trough parallel-ssh. Instead of waiting for complains, one can define metrics to check and use some monitoring appliance for verification. I case of failure remove package from repository and remove-install again. Third will be parallel-ssh install on all the servers. P. S. In case of few tens of servers I'd prefer to work with ansible or alternative, it's worh it in most cases/ Best Regards, Evgeniy. On Tue, Aug 2, 2016 at 8:50 PM, Elazar Leibovich wrote: > Hi, > > I'm having a few (say, a few tens) Debian machines, with a local > repository defined. > > In the local repository I have some home made packages I'm building and > pushing to the local repository. > > When I'm upgrading my package, I want to be sure the update wouldn't cause > a problem. > > So I wish to install them on a few percentage of the machines, wait for > complaints. > > If complaints arrive - roll back. > Otherwise keep upgrading the whole machines. > > I'll appreciate your advice and experience of similar situation, > I'll appreciate if someone who had actual real life experience with this > situation would mention it in the comments. > > Thanks, > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -- So long, and thanks for all the fish. -------------- next part -------------- An HTML attachment was scrubbed... URL: From elazarl at gmail.com Wed Aug 3 09:55:04 2016 From: elazarl at gmail.com (Elazar Leibovich) Date: Wed, 3 Aug 2016 09:55:04 +0300 Subject: Gradual installation of debian packages In-Reply-To: References: Message-ID: How exactly you connect to the server is not in the scope of the discussion, and I agree that ansible is a sensible solution. But what you're proposing is to manually update the package on a small percent of the machines. Manual solution is fine, but I would like to hear experience of people who actually did that on many servers. There are many other issues, for example, how to you roll back? apt-get remove exposes you to the risk that the uninstallation script would be buggy. There are other solutions, e.g., btrfs snapshots on root partitions, but I'm curious to hear someone experienced with it to expose issues I didn't even thought of. Another issue is, how do you select the servers you try it? You suggested a static "beta" list, and I think it's better to select the candidates randomly on each update. Anyhow, how exactly you connect to the server is not the essence of the issue. On Wed, Aug 3, 2016 at 9:30 AM, Evgeniy Ginzburg wrote: > Hello. > I'm assuming that you have paswordless ssh to the servers in question as > root. > Also I assume that you don't use central management/deployment software > (ansible/puppet/chef) > In similar cases I usully use parallel-ssh (gnu-parallel is another > alternative). > First stage install the package manually on one server to see that > configuration is OK, daemons restart, etc... > If this stage is ok second step will be creating list of servers for > "complain" list and install package on them trough parallel-ssh. > Instead of waiting for complains, one can define metrics to check and use > some monitoring appliance for verification. > I case of failure remove package from repository and remove-install again. > Third will be parallel-ssh install on all the servers. > > P. S. In case of few tens of servers I'd prefer to work with ansible or > alternative, it's worh it in most cases/ > > Best Regards, Evgeniy. > > > On Tue, Aug 2, 2016 at 8:50 PM, Elazar Leibovich > wrote: > >> Hi, >> >> I'm having a few (say, a few tens) Debian machines, with a local >> repository defined. >> >> In the local repository I have some home made packages I'm building and >> pushing to the local repository. >> >> When I'm upgrading my package, I want to be sure the update wouldn't >> cause a problem. >> >> So I wish to install them on a few percentage of the machines, wait for >> complaints. >> >> If complaints arrive - roll back. >> Otherwise keep upgrading the whole machines. >> >> I'll appreciate your advice and experience of similar situation, >> I'll appreciate if someone who had actual real life experience with this >> situation would mention it in the comments. >> >> Thanks, >> >> _______________________________________________ >> Linux-il mailing list >> Linux-il at cs.huji.ac.il >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >> >> > > > -- > So long, and thanks for all the fish. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shlomif at gmail.com Wed Aug 3 15:23:04 2016 From: shlomif at gmail.com (Shlomi Fish) Date: Wed, 3 Aug 2016 15:23:04 +0300 Subject: How can the total memory consumption rise without top/htop displaying increasing processes' use? Message-ID: Hi all! I reported a bug in VLC earlier today about it causing increasing total RAM consumption: https://trac.videolan.org/vlc/ticket/17241 However, the strange thing is when it happens, the %MEM usage of the system increases, but I don't see it in htop/top (as root)'s individual processes' RAM consumption. And the memory gets freed after I quit VLC Player. Why? How is it possible? Where does all the memory go? I'm on mageia v6 x86-64 with : shlomif at telaviv1:~$ uname -a Linux telaviv1.shlomifish.org 4.7.0-desktop-2.mga6 #1 SMP Sat Jul 30 21:54:30 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Is this a bug somewhere? Any insights will be appreciated. Regards, -- Shlomi Fish -- Shlomi Fish http://www.shlomifish.org/ You can never truly appreciate The Gilmore Girls until you've watched it in the original Klingon. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From esr+linux-il at g.jct.ac.il Wed Aug 3 15:37:25 2016 From: esr+linux-il at g.jct.ac.il (E.S. Rosenberg) Date: Wed, 3 Aug 2016 15:37:25 +0300 Subject: How can the total memory consumption rise without top/htop displaying increasing processes' use? In-Reply-To: References: Message-ID: This is just a hunch but I think filesystem caching is not counted towards a process' memory usage, so if you are playing a large movie a large chunk (or the whole file) may be kept in memory by the filesystem driver while being memory that is 'available' for immediate freeing it is counted as %used but not as process used since it's the driver/kernel. 2016-08-03 15:23 GMT+03:00 Shlomi Fish : > Hi all! > > I reported a bug in VLC earlier today about it causing increasing total RAM > consumption: > > https://trac.videolan.org/vlc/ticket/17241 > > However, the strange thing is when it happens, the %MEM usage of the system > increases, but I don't see it in htop/top (as root)'s individual processes' > RAM consumption. And the memory gets freed after I quit VLC Player. > > Why? How is it possible? Where does all the memory go? > > I'm on mageia v6 x86-64 with : > > shlomif at telaviv1:~$ uname -a > Linux telaviv1.shlomifish.org 4.7.0-desktop-2.mga6 #1 SMP Sat Jul 30 > 21:54:30 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux > > Is this a bug somewhere? Any insights will be appreciated. > > Regards, > > -- Shlomi Fish > > -- > Shlomi Fish http://www.shlomifish.org/ > > You can never truly appreciate The Gilmore Girls until you've watched it in > the original Klingon. > > Please reply to list if it's a mailing list post - http://shlom.in/reply . > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > From shlomif at gmail.com Wed Aug 3 15:50:42 2016 From: shlomif at gmail.com (Shlomi Fish) Date: Wed, 3 Aug 2016 15:50:42 +0300 Subject: How can the total memory consumption rise without top/htop displaying increasing processes' use? In-Reply-To: References: Message-ID: Hi E. S., On Wed, Aug 3, 2016 at 3:37 PM, E.S. Rosenberg wrote: > This is just a hunch but I think filesystem caching is not counted > towards a process' memory usage, so if you are playing a large movie a > large chunk (or the whole file) may be kept in memory by the > filesystem driver while being memory that is 'available' for immediate > freeing it is counted as %used but not as process used since it's the > driver/kernel. > > Thanks for your reply. However, it also happens to me when playing a file that is 28,407,200 bytes in size while I have over 7 GB of RAM and it consumes hundreds and thousands of megabytes. So it seems unlikely that this is the case here. That put aside, I now have an idea to run vlc under strace and see what it can reveal - so thank you. Regards, -- Shlomi Fish -- Shlomi Fish http://www.shlomifish.org/ You can never truly appreciate The Gilmore Girls until you've watched it in the original Klingon. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From shlomif at gmail.com Wed Aug 3 16:43:45 2016 From: shlomif at gmail.com (Shlomi Fish) Date: Wed, 3 Aug 2016 16:43:45 +0300 Subject: How can the total memory consumption rise without top/htop displaying increasing processes' use? In-Reply-To: References: Message-ID: Hi all, On Wed, Aug 3, 2016 at 3:23 PM, Shlomi Fish wrote: > Hi all! > > I reported a bug in VLC earlier today about it causing increasing total > RAM consumption: > > https://trac.videolan.org/vlc/ticket/17241 > > However, the strange thing is when it happens, the %MEM usage of the > system increases, but I don't see it in htop/top (as root)'s individual > processes' RAM consumption. And the memory gets freed after I quit VLC > Player. > > Why? How is it possible? Where does all the memory go? > > I'm on mageia v6 x86-64 with : > > shlomif at telaviv1:~$ uname -a > Linux telaviv1.shlomifish.org 4.7.0-desktop-2.mga6 #1 SMP Sat Jul 30 > 21:54:30 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux > > Is this a bug somewhere? Any insights will be appreciated. > > One update is that I discovered that the VLC process's "VIRT" column in htop continuously increases while it is exhibiting this problem while the rest of the memory-related columns (including the percentage) remain the same. M_SIZE (VIRT) The size of the virtual memory of the process. Thanks to someone on Freenode for the tip. -- Shlomi Fish http://www.shlomifish.org/ You can never truly appreciate The Gilmore Girls until you've watched it in the original Klingon. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From rabin at rabin.io Wed Aug 3 22:54:52 2016 From: rabin at rabin.io (Rabin Yasharzadehe) Date: Wed, 3 Aug 2016 22:54:52 +0300 Subject: How can the total memory consumption rise without top/htop displaying increasing processes' use? In-Reply-To: References: Message-ID: Maybe it's writing/logging something to a tmpfs folder (like /tmp) -- Rabin On 3 August 2016 at 16:43, Shlomi Fish wrote: > Hi all, > > On Wed, Aug 3, 2016 at 3:23 PM, Shlomi Fish wrote: > >> Hi all! >> >> I reported a bug in VLC earlier today about it causing increasing total >> RAM consumption: >> >> https://trac.videolan.org/vlc/ticket/17241 >> >> However, the strange thing is when it happens, the %MEM usage of the >> system increases, but I don't see it in htop/top (as root)'s individual >> processes' RAM consumption. And the memory gets freed after I quit VLC >> Player. >> >> Why? How is it possible? Where does all the memory go? >> >> I'm on mageia v6 x86-64 with : >> >> shlomif at telaviv1:~$ uname -a >> Linux telaviv1.shlomifish.org 4.7.0-desktop-2.mga6 #1 SMP Sat Jul 30 >> 21:54:30 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux >> >> Is this a bug somewhere? Any insights will be appreciated. >> >> > One update is that I discovered that the VLC process's "VIRT" column in > htop continuously increases while it is exhibiting this problem while the > rest of the memory-related columns (including the percentage) remain the > same. > > M_SIZE (VIRT) > The size of the virtual memory of the process. > > > Thanks to someone on Freenode for the tip. > > -- > Shlomi Fish http://www.shlomifish.org/ > > You can never truly appreciate The Gilmore Girls until you've watched it > in the original Klingon. > > Please reply to list if it's a mailing list post - http://shlom.in/reply . > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amos.shapira at gmail.com Fri Aug 5 15:15:51 2016 From: amos.shapira at gmail.com (Amos Shapira) Date: Fri, 5 Aug 2016 22:15:51 +1000 Subject: Gradual installation of debian packages In-Reply-To: References: Message-ID: What provisioning tools do you use to manage these servers? Please tell me you aren't doing all of this manually. Also what's your environment? All hardware servers? Any virtualisation involved? Cloud servers? Reading your question it feels like you are setting yourself up to fail instead of minimising the failure altogether. What I suggest is that you test your package automatically in a test environment (to me, Vagrant + Rspec/ServerSpec would be first candidates to check) then rollout the package to the repository for the servers to pick it up. As for "roll-back" - with comprehensive automatic testing this concept is becoming obsolete, there is no such thing as "roll-back" only "roll-forward", i.e. since the testing and rolling out are small and "cheap", it should be feasible to fix whatever problem was found instead of having to revert the change altogether. If you are in a properly supported virtual environment then I'd even go for immutable server images (e.g. Packer building AMI's, or Docker containers), then it's a matter of just firing up an instance of the new image both when testing and in production. --Amos On 3 August 2016 at 16:55, Elazar Leibovich wrote: > How exactly you connect to the server is not in the scope of the > discussion, and I agree that ansible is a sensible solution. > > But what you're proposing is to manually update the package on a small > percent of the machines. > > Manual solution is fine, but I would like to hear experience of people who > actually did that on many servers. > > There are many other issues, for example, how to you roll back? > > apt-get remove exposes you to the risk that the uninstallation script > would be buggy. There are other solutions, e.g., btrfs snapshots on root > partitions, but I'm curious to hear someone experienced with it to expose > issues I didn't even thought of. > > Another issue is, how do you select the servers you try it? > > You suggested a static "beta" list, and I think it's better to select the > candidates randomly on each update. > > Anyhow, how exactly you connect to the server is not the essence of the > issue. > > On Wed, Aug 3, 2016 at 9:30 AM, Evgeniy Ginzburg > wrote: > >> Hello. >> I'm assuming that you have paswordless ssh to the servers in question as >> root. >> Also I assume that you don't use central management/deployment software >> (ansible/puppet/chef) >> In similar cases I usully use parallel-ssh (gnu-parallel is another >> alternative). >> First stage install the package manually on one server to see that >> configuration is OK, daemons restart, etc... >> If this stage is ok second step will be creating list of servers for >> "complain" list and install package on them trough parallel-ssh. >> Instead of waiting for complains, one can define metrics to check and use >> some monitoring appliance for verification. >> I case of failure remove package from repository and remove-install again. >> Third will be parallel-ssh install on all the servers. >> >> P. S. In case of few tens of servers I'd prefer to work with ansible or >> alternative, it's worh it in most cases/ >> >> Best Regards, Evgeniy. >> >> >> On Tue, Aug 2, 2016 at 8:50 PM, Elazar Leibovich >> wrote: >> >>> Hi, >>> >>> I'm having a few (say, a few tens) Debian machines, with a local >>> repository defined. >>> >>> In the local repository I have some home made packages I'm building and >>> pushing to the local repository. >>> >>> When I'm upgrading my package, I want to be sure the update wouldn't >>> cause a problem. >>> >>> So I wish to install them on a few percentage of the machines, wait for >>> complaints. >>> >>> If complaints arrive - roll back. >>> Otherwise keep upgrading the whole machines. >>> >>> I'll appreciate your advice and experience of similar situation, >>> I'll appreciate if someone who had actual real life experience with this >>> situation would mention it in the comments. >>> >>> Thanks, >>> >>> _______________________________________________ >>> Linux-il mailing list >>> Linux-il at cs.huji.ac.il >>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>> >>> >> >> >> -- >> So long, and thanks for all the fish. >> > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From elazarl at gmail.com Fri Aug 5 19:14:56 2016 From: elazarl at gmail.com (Elazar Leibovich) Date: Fri, 5 Aug 2016 19:14:56 +0300 Subject: Gradual installation of debian packages In-Reply-To: References: Message-ID: All real servers, with custom hardware attached, geographically distributed across the planet. Real people actually use the hardware attached to this computers, and it's not obvious to test whether or not it failed. The strategy therefor is, deploy randomly to small percentage of the machines, wait to see if you get complains from those customers using these hardware devices, and if everything went well, update the rest of the servers. The provisioning solution is chef, but I'm open to changing it. As I said, I don't think it makes too much difference. As of immutable server images, I'd do it with ZFS/brtfs snapshots (+docker/machinectl/systemd-nspawn if you must have some sort of virtual environment), but it's probably a better idea than apt-get install pkg=oldversion. Immutable filesystem for execution is of course not enough, since you might have migrations for the mutable part, etc. In this particular case, I don't think it's a big deal. You see, not everything is a web startup with customer facing website ;-) Thanks, Appreciate you sharing your experience. I'm not disagreeing with your points, but in this particular case, where testing is expensive, not all of them seems valid. On Fri, Aug 5, 2016 at 3:15 PM, Amos Shapira wrote: > What provisioning tools do you use to manage these servers? Please tell me > you aren't doing all of this manually. > Also what's your environment? All hardware servers? Any virtualisation > involved? Cloud servers? > > Reading your question it feels like you are setting yourself up to fail > instead of minimising the failure altogether. > > What I suggest is that you test your package automatically in a test > environment (to me, Vagrant + Rspec/ServerSpec would be first candidates to > check) then rollout the package to the repository for the servers to pick > it up. > > As for "roll-back" - with comprehensive automatic testing this concept is > becoming obsolete, there is no such thing as "roll-back" only > "roll-forward", i.e. since the testing and rolling out are small and > "cheap", it should be feasible to fix whatever problem was found instead of > having to revert the change altogether. > > If you are in a properly supported virtual environment then I'd even go > for immutable server images (e.g. Packer building AMI's, or Docker > containers), then it's a matter of just firing up an instance of the new > image both when testing and in production. > > --Amos > > On 3 August 2016 at 16:55, Elazar Leibovich wrote: > >> How exactly you connect to the server is not in the scope of the >> discussion, and I agree that ansible is a sensible solution. >> >> But what you're proposing is to manually update the package on a small >> percent of the machines. >> >> Manual solution is fine, but I would like to hear experience of people >> who actually did that on many servers. >> >> There are many other issues, for example, how to you roll back? >> >> apt-get remove exposes you to the risk that the uninstallation script >> would be buggy. There are other solutions, e.g., btrfs snapshots on root >> partitions, but I'm curious to hear someone experienced with it to expose >> issues I didn't even thought of. >> >> Another issue is, how do you select the servers you try it? >> >> You suggested a static "beta" list, and I think it's better to select the >> candidates randomly on each update. >> >> Anyhow, how exactly you connect to the server is not the essence of the >> issue. >> >> On Wed, Aug 3, 2016 at 9:30 AM, Evgeniy Ginzburg >> wrote: >> >>> Hello. >>> I'm assuming that you have paswordless ssh to the servers in question as >>> root. >>> Also I assume that you don't use central management/deployment software >>> (ansible/puppet/chef) >>> In similar cases I usully use parallel-ssh (gnu-parallel is another >>> alternative). >>> First stage install the package manually on one server to see that >>> configuration is OK, daemons restart, etc... >>> If this stage is ok second step will be creating list of servers for >>> "complain" list and install package on them trough parallel-ssh. >>> Instead of waiting for complains, one can define metrics to check and >>> use some monitoring appliance for verification. >>> I case of failure remove package from repository and remove-install >>> again. >>> Third will be parallel-ssh install on all the servers. >>> >>> P. S. In case of few tens of servers I'd prefer to work with ansible or >>> alternative, it's worh it in most cases/ >>> >>> Best Regards, Evgeniy. >>> >>> >>> On Tue, Aug 2, 2016 at 8:50 PM, Elazar Leibovich >>> wrote: >>> >>>> Hi, >>>> >>>> I'm having a few (say, a few tens) Debian machines, with a local >>>> repository defined. >>>> >>>> In the local repository I have some home made packages I'm building and >>>> pushing to the local repository. >>>> >>>> When I'm upgrading my package, I want to be sure the update wouldn't >>>> cause a problem. >>>> >>>> So I wish to install them on a few percentage of the machines, wait for >>>> complaints. >>>> >>>> If complaints arrive - roll back. >>>> Otherwise keep upgrading the whole machines. >>>> >>>> I'll appreciate your advice and experience of similar situation, >>>> I'll appreciate if someone who had actual real life experience with >>>> this situation would mention it in the comments. >>>> >>>> Thanks, >>>> >>>> _______________________________________________ >>>> Linux-il mailing list >>>> Linux-il at cs.huji.ac.il >>>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>>> >>>> >>> >>> >>> -- >>> So long, and thanks for all the fish. >>> >> >> >> _______________________________________________ >> Linux-il mailing list >> Linux-il at cs.huji.ac.il >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >> >> > > > -- > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amos.shapira at gmail.com Sat Aug 6 02:27:09 2016 From: amos.shapira at gmail.com (Amos Shapira) Date: Sat, 6 Aug 2016 09:27:09 +1000 Subject: Gradual installation of debian packages In-Reply-To: References: Message-ID: What kind of hardware is this that's connected to the servers, and what does the software do that you can't test before installing on production servers? On 6 August 2016 at 02:14, Elazar Leibovich wrote: > All real servers, with custom hardware attached, geographically > distributed across the planet. > > Real people actually use the hardware attached to this computers, and it's > not obvious to test whether or not it failed. > > The strategy therefor is, deploy randomly to small percentage of the > machines, wait to see if you get complains from those customers using these > hardware devices, and if everything went well, update the rest of the > servers. > > The provisioning solution is chef, but I'm open to changing it. As I said, > I don't think it makes too much difference. > > As of immutable server images, I'd do it with ZFS/brtfs snapshots > (+docker/machinectl/systemd-nspawn if you must have some sort of virtual > environment), but it's probably a better idea than apt-get install > pkg=oldversion. Immutable filesystem for execution is of course not enough, > since you might have migrations for the mutable part, etc. In this > particular case, I don't think it's a big deal. > > You see, not everything is a web startup with customer facing website ;-) > > Thanks, > Appreciate you sharing your experience. > I'm not disagreeing with your points, but in this particular case, where > testing is expensive, not all of them seems valid. > > On Fri, Aug 5, 2016 at 3:15 PM, Amos Shapira > wrote: > >> What provisioning tools do you use to manage these servers? Please tell >> me you aren't doing all of this manually. >> Also what's your environment? All hardware servers? Any virtualisation >> involved? Cloud servers? >> >> Reading your question it feels like you are setting yourself up to fail >> instead of minimising the failure altogether. >> >> What I suggest is that you test your package automatically in a test >> environment (to me, Vagrant + Rspec/ServerSpec would be first candidates to >> check) then rollout the package to the repository for the servers to pick >> it up. >> >> As for "roll-back" - with comprehensive automatic testing this concept is >> becoming obsolete, there is no such thing as "roll-back" only >> "roll-forward", i.e. since the testing and rolling out are small and >> "cheap", it should be feasible to fix whatever problem was found instead of >> having to revert the change altogether. >> >> If you are in a properly supported virtual environment then I'd even go >> for immutable server images (e.g. Packer building AMI's, or Docker >> containers), then it's a matter of just firing up an instance of the new >> image both when testing and in production. >> >> --Amos >> >> On 3 August 2016 at 16:55, Elazar Leibovich wrote: >> >>> How exactly you connect to the server is not in the scope of the >>> discussion, and I agree that ansible is a sensible solution. >>> >>> But what you're proposing is to manually update the package on a small >>> percent of the machines. >>> >>> Manual solution is fine, but I would like to hear experience of people >>> who actually did that on many servers. >>> >>> There are many other issues, for example, how to you roll back? >>> >>> apt-get remove exposes you to the risk that the uninstallation script >>> would be buggy. There are other solutions, e.g., btrfs snapshots on root >>> partitions, but I'm curious to hear someone experienced with it to expose >>> issues I didn't even thought of. >>> >>> Another issue is, how do you select the servers you try it? >>> >>> You suggested a static "beta" list, and I think it's better to select >>> the candidates randomly on each update. >>> >>> Anyhow, how exactly you connect to the server is not the essence of the >>> issue. >>> >>> On Wed, Aug 3, 2016 at 9:30 AM, Evgeniy Ginzburg >>> wrote: >>> >>>> Hello. >>>> I'm assuming that you have paswordless ssh to the servers in question >>>> as root. >>>> Also I assume that you don't use central management/deployment software >>>> (ansible/puppet/chef) >>>> In similar cases I usully use parallel-ssh (gnu-parallel is another >>>> alternative). >>>> First stage install the package manually on one server to see that >>>> configuration is OK, daemons restart, etc... >>>> If this stage is ok second step will be creating list of servers for >>>> "complain" list and install package on them trough parallel-ssh. >>>> Instead of waiting for complains, one can define metrics to check and >>>> use some monitoring appliance for verification. >>>> I case of failure remove package from repository and remove-install >>>> again. >>>> Third will be parallel-ssh install on all the servers. >>>> >>>> P. S. In case of few tens of servers I'd prefer to work with ansible or >>>> alternative, it's worh it in most cases/ >>>> >>>> Best Regards, Evgeniy. >>>> >>>> >>>> On Tue, Aug 2, 2016 at 8:50 PM, Elazar Leibovich >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm having a few (say, a few tens) Debian machines, with a local >>>>> repository defined. >>>>> >>>>> In the local repository I have some home made packages I'm building >>>>> and pushing to the local repository. >>>>> >>>>> When I'm upgrading my package, I want to be sure the update wouldn't >>>>> cause a problem. >>>>> >>>>> So I wish to install them on a few percentage of the machines, wait >>>>> for complaints. >>>>> >>>>> If complaints arrive - roll back. >>>>> Otherwise keep upgrading the whole machines. >>>>> >>>>> I'll appreciate your advice and experience of similar situation, >>>>> I'll appreciate if someone who had actual real life experience with >>>>> this situation would mention it in the comments. >>>>> >>>>> Thanks, >>>>> >>>>> _______________________________________________ >>>>> Linux-il mailing list >>>>> Linux-il at cs.huji.ac.il >>>>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>>>> >>>>> >>>> >>>> >>>> -- >>>> So long, and thanks for all the fish. >>>> >>> >>> >>> _______________________________________________ >>> Linux-il mailing list >>> Linux-il at cs.huji.ac.il >>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>> >>> >> >> >> -- >> >> > > -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From elazarl at gmail.com Sun Aug 7 07:18:54 2016 From: elazarl at gmail.com (Elazar Leibovich) Date: Sun, 7 Aug 2016 07:18:54 +0300 Subject: Gradual installation of debian packages In-Reply-To: References: Message-ID: It's radio antenna. It is of course tested before to some extent, in a "staging" environment. But since the physical environment varies, and sometimes antenna related parameters change between releases (e.g., duration of receive time), it is not easy to know you're not breaking something for someone by mistake. It could be for example the physical location of the antenna at the client which would make a difference. On Sat, Aug 6, 2016 at 2:27 AM, Amos Shapira wrote: > What kind of hardware is this that's connected to the servers, and what > does the software do that you can't test before installing on production > servers? > > On 6 August 2016 at 02:14, Elazar Leibovich wrote: > >> All real servers, with custom hardware attached, geographically >> distributed across the planet. >> >> Real people actually use the hardware attached to this computers, and >> it's not obvious to test whether or not it failed. >> >> The strategy therefor is, deploy randomly to small percentage of the >> machines, wait to see if you get complains from those customers using these >> hardware devices, and if everything went well, update the rest of the >> servers. >> >> The provisioning solution is chef, but I'm open to changing it. As I >> said, I don't think it makes too much difference. >> >> As of immutable server images, I'd do it with ZFS/brtfs snapshots >> (+docker/machinectl/systemd-nspawn if you must have some sort of virtual >> environment), but it's probably a better idea than apt-get install >> pkg=oldversion. Immutable filesystem for execution is of course not enough, >> since you might have migrations for the mutable part, etc. In this >> particular case, I don't think it's a big deal. >> >> You see, not everything is a web startup with customer facing website ;-) >> >> Thanks, >> Appreciate you sharing your experience. >> I'm not disagreeing with your points, but in this particular case, where >> testing is expensive, not all of them seems valid. >> >> On Fri, Aug 5, 2016 at 3:15 PM, Amos Shapira >> wrote: >> >>> What provisioning tools do you use to manage these servers? Please tell >>> me you aren't doing all of this manually. >>> Also what's your environment? All hardware servers? Any virtualisation >>> involved? Cloud servers? >>> >>> Reading your question it feels like you are setting yourself up to fail >>> instead of minimising the failure altogether. >>> >>> What I suggest is that you test your package automatically in a test >>> environment (to me, Vagrant + Rspec/ServerSpec would be first candidates to >>> check) then rollout the package to the repository for the servers to pick >>> it up. >>> >>> As for "roll-back" - with comprehensive automatic testing this concept >>> is becoming obsolete, there is no such thing as "roll-back" only >>> "roll-forward", i.e. since the testing and rolling out are small and >>> "cheap", it should be feasible to fix whatever problem was found instead of >>> having to revert the change altogether. >>> >>> If you are in a properly supported virtual environment then I'd even go >>> for immutable server images (e.g. Packer building AMI's, or Docker >>> containers), then it's a matter of just firing up an instance of the new >>> image both when testing and in production. >>> >>> --Amos >>> >>> On 3 August 2016 at 16:55, Elazar Leibovich wrote: >>> >>>> How exactly you connect to the server is not in the scope of the >>>> discussion, and I agree that ansible is a sensible solution. >>>> >>>> But what you're proposing is to manually update the package on a small >>>> percent of the machines. >>>> >>>> Manual solution is fine, but I would like to hear experience of people >>>> who actually did that on many servers. >>>> >>>> There are many other issues, for example, how to you roll back? >>>> >>>> apt-get remove exposes you to the risk that the uninstallation script >>>> would be buggy. There are other solutions, e.g., btrfs snapshots on root >>>> partitions, but I'm curious to hear someone experienced with it to expose >>>> issues I didn't even thought of. >>>> >>>> Another issue is, how do you select the servers you try it? >>>> >>>> You suggested a static "beta" list, and I think it's better to select >>>> the candidates randomly on each update. >>>> >>>> Anyhow, how exactly you connect to the server is not the essence of the >>>> issue. >>>> >>>> On Wed, Aug 3, 2016 at 9:30 AM, Evgeniy Ginzburg >>>> wrote: >>>> >>>>> Hello. >>>>> I'm assuming that you have paswordless ssh to the servers in question >>>>> as root. >>>>> Also I assume that you don't use central management/deployment >>>>> software (ansible/puppet/chef) >>>>> In similar cases I usully use parallel-ssh (gnu-parallel is another >>>>> alternative). >>>>> First stage install the package manually on one server to see that >>>>> configuration is OK, daemons restart, etc... >>>>> If this stage is ok second step will be creating list of servers for >>>>> "complain" list and install package on them trough parallel-ssh. >>>>> Instead of waiting for complains, one can define metrics to check and >>>>> use some monitoring appliance for verification. >>>>> I case of failure remove package from repository and remove-install >>>>> again. >>>>> Third will be parallel-ssh install on all the servers. >>>>> >>>>> P. S. In case of few tens of servers I'd prefer to work with ansible >>>>> or alternative, it's worh it in most cases/ >>>>> >>>>> Best Regards, Evgeniy. >>>>> >>>>> >>>>> On Tue, Aug 2, 2016 at 8:50 PM, Elazar Leibovich >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm having a few (say, a few tens) Debian machines, with a local >>>>>> repository defined. >>>>>> >>>>>> In the local repository I have some home made packages I'm building >>>>>> and pushing to the local repository. >>>>>> >>>>>> When I'm upgrading my package, I want to be sure the update wouldn't >>>>>> cause a problem. >>>>>> >>>>>> So I wish to install them on a few percentage of the machines, wait >>>>>> for complaints. >>>>>> >>>>>> If complaints arrive - roll back. >>>>>> Otherwise keep upgrading the whole machines. >>>>>> >>>>>> I'll appreciate your advice and experience of similar situation, >>>>>> I'll appreciate if someone who had actual real life experience with >>>>>> this situation would mention it in the comments. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> _______________________________________________ >>>>>> Linux-il mailing list >>>>>> Linux-il at cs.huji.ac.il >>>>>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> So long, and thanks for all the fish. >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Linux-il mailing list >>>> Linux-il at cs.huji.ac.il >>>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>>> >>>> >>> >>> >>> -- >>> >>> >> >> > > > -- > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amos.shapira at gmail.com Sun Aug 7 07:53:01 2016 From: amos.shapira at gmail.com (Amos Shapira) Date: Sun, 7 Aug 2016 14:53:01 +1000 Subject: Gradual installation of debian packages In-Reply-To: References: Message-ID: I see. Valid points. Whenever you break a production site - do you try to add a test which simulates the parameters of the breakage? It sounds to me like some sort of an image versioning could still help here, that way you can really "roll back" (actually boot to a previous version of the image) properly. For instance, VyOS (http://vyos.net/wiki/Upgrade) roll out new versions this way. I'm not sure how exactly they do that but the bottom line is that it's possible to upgrade to the next release and still save all the versions and configuration to roll back if you have to. On 7 August 2016 at 14:18, Elazar Leibovich wrote: > It's radio antenna. > > It is of course tested before to some extent, in a "staging" environment. > > But since the physical environment varies, and sometimes antenna related > parameters change between releases (e.g., duration of receive time), it is > not easy to know you're not breaking something for someone by mistake. > > It could be for example the physical location of the antenna at the client > which would make a difference. > > > On Sat, Aug 6, 2016 at 2:27 AM, Amos Shapira > wrote: > >> What kind of hardware is this that's connected to the servers, and what >> does the software do that you can't test before installing on production >> servers? >> >> On 6 August 2016 at 02:14, Elazar Leibovich wrote: >> >>> All real servers, with custom hardware attached, geographically >>> distributed across the planet. >>> >>> Real people actually use the hardware attached to this computers, and >>> it's not obvious to test whether or not it failed. >>> >>> The strategy therefor is, deploy randomly to small percentage of the >>> machines, wait to see if you get complains from those customers using these >>> hardware devices, and if everything went well, update the rest of the >>> servers. >>> >>> The provisioning solution is chef, but I'm open to changing it. As I >>> said, I don't think it makes too much difference. >>> >>> As of immutable server images, I'd do it with ZFS/brtfs snapshots >>> (+docker/machinectl/systemd-nspawn if you must have some sort of >>> virtual environment), but it's probably a better idea than apt-get install >>> pkg=oldversion. Immutable filesystem for execution is of course not enough, >>> since you might have migrations for the mutable part, etc. In this >>> particular case, I don't think it's a big deal. >>> >>> You see, not everything is a web startup with customer facing website ;-) >>> >>> Thanks, >>> Appreciate you sharing your experience. >>> I'm not disagreeing with your points, but in this particular case, where >>> testing is expensive, not all of them seems valid. >>> >>> On Fri, Aug 5, 2016 at 3:15 PM, Amos Shapira >>> wrote: >>> >>>> What provisioning tools do you use to manage these servers? Please tell >>>> me you aren't doing all of this manually. >>>> Also what's your environment? All hardware servers? Any virtualisation >>>> involved? Cloud servers? >>>> >>>> Reading your question it feels like you are setting yourself up to fail >>>> instead of minimising the failure altogether. >>>> >>>> What I suggest is that you test your package automatically in a test >>>> environment (to me, Vagrant + Rspec/ServerSpec would be first candidates to >>>> check) then rollout the package to the repository for the servers to pick >>>> it up. >>>> >>>> As for "roll-back" - with comprehensive automatic testing this concept >>>> is becoming obsolete, there is no such thing as "roll-back" only >>>> "roll-forward", i.e. since the testing and rolling out are small and >>>> "cheap", it should be feasible to fix whatever problem was found instead of >>>> having to revert the change altogether. >>>> >>>> If you are in a properly supported virtual environment then I'd even go >>>> for immutable server images (e.g. Packer building AMI's, or Docker >>>> containers), then it's a matter of just firing up an instance of the new >>>> image both when testing and in production. >>>> >>>> --Amos >>>> >>>> On 3 August 2016 at 16:55, Elazar Leibovich wrote: >>>> >>>>> How exactly you connect to the server is not in the scope of the >>>>> discussion, and I agree that ansible is a sensible solution. >>>>> >>>>> But what you're proposing is to manually update the package on a small >>>>> percent of the machines. >>>>> >>>>> Manual solution is fine, but I would like to hear experience of people >>>>> who actually did that on many servers. >>>>> >>>>> There are many other issues, for example, how to you roll back? >>>>> >>>>> apt-get remove exposes you to the risk that the uninstallation script >>>>> would be buggy. There are other solutions, e.g., btrfs snapshots on root >>>>> partitions, but I'm curious to hear someone experienced with it to expose >>>>> issues I didn't even thought of. >>>>> >>>>> Another issue is, how do you select the servers you try it? >>>>> >>>>> You suggested a static "beta" list, and I think it's better to select >>>>> the candidates randomly on each update. >>>>> >>>>> Anyhow, how exactly you connect to the server is not the essence of >>>>> the issue. >>>>> >>>>> On Wed, Aug 3, 2016 at 9:30 AM, Evgeniy Ginzburg >>>>> wrote: >>>>> >>>>>> Hello. >>>>>> I'm assuming that you have paswordless ssh to the servers in question >>>>>> as root. >>>>>> Also I assume that you don't use central management/deployment >>>>>> software (ansible/puppet/chef) >>>>>> In similar cases I usully use parallel-ssh (gnu-parallel is another >>>>>> alternative). >>>>>> First stage install the package manually on one server to see that >>>>>> configuration is OK, daemons restart, etc... >>>>>> If this stage is ok second step will be creating list of servers for >>>>>> "complain" list and install package on them trough parallel-ssh. >>>>>> Instead of waiting for complains, one can define metrics to check and >>>>>> use some monitoring appliance for verification. >>>>>> I case of failure remove package from repository and remove-install >>>>>> again. >>>>>> Third will be parallel-ssh install on all the servers. >>>>>> >>>>>> P. S. In case of few tens of servers I'd prefer to work with ansible >>>>>> or alternative, it's worh it in most cases/ >>>>>> >>>>>> Best Regards, Evgeniy. >>>>>> >>>>>> >>>>>> On Tue, Aug 2, 2016 at 8:50 PM, Elazar Leibovich >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm having a few (say, a few tens) Debian machines, with a local >>>>>>> repository defined. >>>>>>> >>>>>>> In the local repository I have some home made packages I'm building >>>>>>> and pushing to the local repository. >>>>>>> >>>>>>> When I'm upgrading my package, I want to be sure the update wouldn't >>>>>>> cause a problem. >>>>>>> >>>>>>> So I wish to install them on a few percentage of the machines, wait >>>>>>> for complaints. >>>>>>> >>>>>>> If complaints arrive - roll back. >>>>>>> Otherwise keep upgrading the whole machines. >>>>>>> >>>>>>> I'll appreciate your advice and experience of similar situation, >>>>>>> I'll appreciate if someone who had actual real life experience with >>>>>>> this situation would mention it in the comments. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Linux-il mailing list >>>>>>> Linux-il at cs.huji.ac.il >>>>>>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> So long, and thanks for all the fish. >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Linux-il mailing list >>>>> Linux-il at cs.huji.ac.il >>>>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> >>> >>> >> >> >> -- >> >> > > -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From uri at speedy.net Tue Aug 23 11:11:05 2016 From: uri at speedy.net (Uri Even-Chen) Date: Tue, 23 Aug 2016 11:11:05 +0300 Subject: Replacing ISOC as my domain registrar In-Reply-To: <61baf180-4297-e4b8-933e-0e3924b2e4d9@mln.co.il> References: <20160726170805.5c314eaa@shlomo1.solomon> <61baf180-4297-e4b8-933e-0e3924b2e4d9@mln.co.il> Message-ID: I worked with LiveDNS , no problems so far. I don't like Interspace. All the others I don't know. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ On Wed, Jul 27, 2016 at 3:17 PM, Moish wrote: > ISOC announced that they will cease to mainatain my domain's regisration. > My dns records are maintained by Google and Netvision. > > How do I select a new registrar? > Whats should I look for? > > > Moish > > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > -------------- next part -------------- An HTML attachment was scrubbed... URL: From const at makelinux.co.il Wed Aug 31 01:09:51 2016 From: const at makelinux.co.il (Constantine Shulyupin) Date: Wed, 31 Aug 2016 01:09:51 +0300 Subject: Errors resolver Message-ID: Hi I develop application, which analyzes and helps to solve various compilation and system errors. For example: echo "warning: implicit declaration of function ?pthread_create?" | ./errors_resolver.py Output with solution: CPPFLAGS+=' -include pthread.h'; Tn this example errors_resolver.py searches tags and provides missing header file. You are welcome to review it and give feedback: https://github.com/makelinux/errors_resolver Thanks -- Constantine Shulyupin http://www.MakeLinux.co.il/ Embedded Linux Systems Tel Aviv