From mulix at mulix.org Tue Aug 4 11:50:21 2015 From: mulix at mulix.org (Muli Ben-Yehuda) Date: Tue, 4 Aug 2015 11:50:21 +0300 Subject: Fwd: compiler/tools/architecture conference Message-ID: <20150804085021.GB1327@needle> Might interest some of you. Apologies if you receive multiple copies. CALL FOR PAPERS Compiler, Architecture and Tools Conference (CATC) Intel Development Center, Haifa, Israel November 23rd, 2015 http://software.intel.com/compilerconf2015 Endorsed by the HiPEAC Network of Excellence ==================================================================== IMPORTANT DATES * Paper Submission deadline: August 6th, 2015 * Author Notification: September 10th, 2015 * Conference date: November 23rd, 2015 ==================================================================== OVERVIEW The interaction among advanced compilation techniques, modern processor and computing architectures, and associated tools continues to face new challenges and opportunities. Traditional demands to increase performance, reduce power consumption, and reduce time to market now apply to heterogeneous, virtualized and diverse user-experience environments. Extensive data and task parallelism are being exposed by new programming environments such as Halide, Swift, and OpenMP version 4.0, relying on innovative architectures, compilers, binary translation and runtime tools. This conference will focus on these exciting new directions and how they are influencing the architecture and compilation domain. The conference will be held in a format similar to that held last year leaving time for networking and presentations. TOPICS The main focus of this conference is the interaction of compiler technologies, processor and computing architectures and tools to address the latest programming environments and demands. The topics of interest for this conference include, but are not limited to: * Compilers, runtime and tools for modern server, client, mobile and embedded systems * Compiler/hardware support for hiding memory and I/O latencies * Compilation to hardware * Dynamic translation and optimization * Heterogeneous parallel architectures and computational models * GPU, accelerator and coprocessor architectures * Power/Performance/Monitoring tools for application behavior understanding * Parallel programming languages, algorithms and applications * Internet of Things INVITED SPEAKERS We are delighted to have Dr. Youfeng Wu, Principal Research Scientist at Intel and Prof. Mooly (Shmuel) Sagiv from Tel-Aviv University as our keynote speakers. VENUE The conference will take place at Intel Development Center (bldg. IDC9) in Matam Industrial Park, Haifa, Israel. Lunch and light refreshments will be served. Participation in the conference is free of charge, but registration is required. The official language of the conference is English. SUBMISSION Authors should submit an abstract of their contributions by email to Gadi Haber (gadi.haber at intel.com). Submissions must include an abstract of 100-300 words and contact information. The authors are encouraged to provide related documents with additional details on their research work, such as an extended abstract, paper, and/or links. Abstracts and eventually presentations will be provided online at the conference webpage. The conference does not include proceedings, however, with author approval, papers may appear online and extended versions of the best papers may be included in a special issue of IJPP. In addition, authors may submit papers recently presented in (or accepted to) top-tier conferences in the areas of interest. ORGANIZERS Ayal Zaks, Dorit Nuzman, Gadi Haber, Leeor Peled, Michal Nir; Intel Development Center, Haifa (first.last at intel.com) Erez Petrank; Computer Science Dept., Technion, Haifa (first at cs.technion.ac.il) Yosi Ben-Asher; Computer Science Dept., University of Haifa (first at cs.haifa.ac.il) PAST CONFERENCE http://software.intel.com/compilerconf2014 From shachar at shemesh.biz Sat Aug 8 20:31:54 2015 From: shachar at shemesh.biz (Shachar Shemesh) Date: Sat, 8 Aug 2015 20:31:54 +0300 Subject: Matrix inversion tool Message-ID: <55C63D0A.7000902@shemesh.biz> Hi all, I'm looking for a tool/code to invert a matrix. So far, this sounds trivial. I have one special requirement. I did not think it was too special, except I could not find anywhere that supplied it. I want the matrix to be over a different field (i.e. - not the real numbers). In my particular case, I want it to be over Z2 (zero and one). So far, I have failed to find such a tool. If anyone knows how to do it, please let me know, as the alternative is writing one myself, which is something I REALLY don't want to do without a good reason. Shachar -------------- next part -------------- An HTML attachment was scrubbed... URL: From w1 at zak.co.il Sat Aug 8 20:56:11 2015 From: w1 at zak.co.il (Omer Zak) Date: Sat, 08 Aug 2015 20:56:11 +0300 Subject: Matrix inversion tool In-Reply-To: <55C63D0A.7000902@shemesh.biz> References: <55C63D0A.7000902@shemesh.biz> Message-ID: <1439056571.10934.54.camel@zak.co.il> What happens if you use the regular matrix inversion tool which works on real numbers? (after rejecting, as singular over Z2, matrices whose determinant modulo 2 is different from 1) On Sat, 2015-08-08 at 20:31 +0300, Shachar Shemesh wrote: > Hi all, > > I'm looking for a tool/code to invert a matrix. So far, this sounds > trivial. I have one special requirement. I did not think it was too > special, except I could not find anywhere that supplied it. > > I want the matrix to be over a different field (i.e. - not the real > numbers). In my particular case, I want it to be over Z2 (zero and > one). > > So far, I have failed to find such a tool. If anyone knows how to do > it, please let me know, as the alternative is writing one myself, > which is something I REALLY don't want to do without a good reason. -- If verbal consent is not obtained in triplicate, it is a date rape. Asking permission constitutes harassment. My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.htmlDelay is the deadliest form of denial. C. Northcote Parkinson My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html From matan at svgalib.org Sat Aug 8 21:07:25 2015 From: matan at svgalib.org (Matan Ziv-Av) Date: Sat, 8 Aug 2015 21:07:25 +0300 (IDT) Subject: Matrix inversion tool In-Reply-To: <1439056571.10934.54.camel@zak.co.il> References: <55C63D0A.7000902@shemesh.biz> <1439056571.10934.54.camel@zak.co.il> Message-ID: On Sat, 8 Aug 2015, Omer Zak wrote: > What happens if you use the regular matrix inversion tool which works on > real numbers? > (after rejecting, as singular over Z2, matrices whose determinant modulo > 2 is different from 1) This will work if the inverse only has integer entries. It may also work if your algorithm returns rational numbers, but will not work with floating point numbers (there are no reals in a computer). You cannot faithfully convert 1.1234 to an element of Z_2. > On Sat, 2015-08-08 at 20:31 +0300, Shachar Shemesh wrote: >> Hi all, >> >> I'm looking for a tool/code to invert a matrix. So far, this sounds >> trivial. I have one special requirement. I did not think it was too >> special, except I could not find anywhere that supplied it. >> >> I want the matrix to be over a different field (i.e. - not the real >> numbers). In my particular case, I want it to be over Z2 (zero and >> one). >> >> So far, I have failed to find such a tool. If anyone knows how to do >> it, please let me know, as the alternative is writing one myself, >> which is something I REALLY don't want to do without a good reason. You can use GAP ( http://www.gap-system.org/ ). > -- Matan. From yyweiss at cs.huji.ac.il Sat Aug 8 21:31:08 2015 From: yyweiss at cs.huji.ac.il (Yaacov Weiss) Date: Sat, 8 Aug 2015 21:31:08 +0300 Subject: Matrix inversion tool In-Reply-To: <55C63D0A.7000902@shemesh.biz> References: <55C63D0A.7000902@shemesh.biz> Message-ID: On Sat, Aug 8, 2015 at 8:31 PM, Shachar Shemesh wrote: > I want the matrix to be over a different field (i.e. - not the real > numbers). In my particular case, I want it to be over Z2 (zero and one). The top results of a search for Finite Field library look promising. From shachar at shemesh.biz Sun Aug 9 08:22:04 2015 From: shachar at shemesh.biz (Shachar Shemesh) Date: Sun, 09 Aug 2015 08:22:04 +0300 Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> Message-ID: <474ed5de0b20bceceb237003f6dbefef@shemesh.biz> On 08/08/2015 21:31, Yaacov Weiss wrote: > On Sat, Aug 8, 2015 at 8:31 PM, Shachar Shemesh wrote: > >> I want the matrix to be over a different field (i.e. - not the real numbers). In my particular case, I want it to be over Z2 (zero and one). > > The top results of a search for Finite Field library look promising. Yes, Givaro seems promising. I did not try this search because most results are for Galois fields (which, granted, are not that different for the specific case of "2", but still). I'll have a look into that. I might eventually need to bind to it from D, which will probably not be fun, but we'll cross that bridge when its time comes. Many people sent suggestions in private. I'll point out that: * I tried using Boost's matrix code with a type that is a special class I wrote. Not only did it not work, I couldn't even decipher what it was that it was missing in order to get it to work. The error messages were 300 characters long and five to six lines long PER ERROR. Even when unravelled, they complain that some internal class used by Boost is not comparable (or something) to some other internal class used by Boost. * My linear algebra isn't as well remembered as I thought it was. When I manually inverse a matrix, I use row replacement, addition and subtraction. I vaguely remember what covariants are, and no recollection at all of adj. I will definitely look them up before I continue, but it's somewhat sad when things you once knew you no longer do. Unrelated to the above, but does anyone have a GOOD source explaining Galois fields? Everything I found so far left me confused about whether it is used directly (i.e. - numbers treated as polynoms, divided by a modulus), or via a generator. Also, I know they are used for Raid-6 calculations, but it seems to me that if I multiply a value by 2, I am going to get an overflow (unless I'm modulus something which isn't even, in which case I cannot represent all 2^32 values of an int). If anyone has any source that clears up that confusion for me, please send me a link. Thanks, Shachar -------------- next part -------------- An HTML attachment was scrubbed... URL: From pub at goldshmidt.org Sun Aug 9 08:37:11 2015 From: pub at goldshmidt.org (Oleg Goldshmidt) Date: Sun, 09 Aug 2015 08:37:11 +0300 Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <1439056571.10934.54.camel@zak.co.il> Message-ID: <87614pm394.fsf@goldshmidt.org> Matan Ziv-Av writes: > On Sat, 8 Aug 2015, Omer Zak wrote: > >> What happens if you use the regular matrix inversion tool which works >> on real numbers? (after rejecting, as singular over Z2, matrices >> whose determinant modulo 2 is different from 1) > > This will work if the inverse only has integer entries. It may also > work if your algorithm returns rational numbers, but will not work > with floating point numbers (there are no reals in a computer). You > cannot faithfully convert 1.1234 to an element of Z_2. The original matrix contains zeros and ones, and you never need to divide, just multiply and add modulo 2. -- Oleg Goldshmidt | pub at goldshmidt.org From pub at goldshmidt.org Sun Aug 9 08:33:19 2015 From: pub at goldshmidt.org (Oleg Goldshmidt) Date: Sun, 09 Aug 2015 08:33:19 +0300 Subject: Matrix inversion tool In-Reply-To: <55C63D0A.7000902@shemesh.biz> References: <55C63D0A.7000902@shemesh.biz> Message-ID: <87bnehm3fk.fsf@goldshmidt.org> Shachar Shemesh writes: > Hi all, > > I'm looking for a tool/code to invert a matrix. So far, this sounds > trivial. I have one special requirement. I did not think it was too > special, except I could not find anywhere that supplied it. > > I want the matrix to be over a different field (i.e. - not the real > numbers). In my particular case, I want it to be over Z2 (zero and > one). If you don't find ready code writing your own should be pretty simple for Z2 - fields don't get any simpler. ;-) In general, if you have a matrix A, then its inverse is given by A^(-1) = adj(A)/det(A) [1] The determinant of any matrix over Z2 is either 0 or 1. If it is 0 the matrix is not invertible. If it is 1, then the inverse matrix is the same as the adjugate of the original matrix. Now, the adjugate is the transpose of the cofactor matrix, which is C_ij = (-1)^(i+j)*M_ij [2] where M_ij is A's (ij)-minor, i.e., the determinant of the matrix derived by deleting the i-th row and the j-th column of A. Thus adj(A) = C_ji = (-1)^(i+j)*M_ji [3] Thus, from equations 1 and 3 we have {A^(-1)}_ij = (-1)^(i+j)*M_ji [4] There are probably library routines to compute minors and cofactors. If you don't find any, then, given that all you have is zeroes and ones, it should be easy and safe to write your own. Note that you need to compute the determinant to determine whether or not your matrix is invertible, and to compute the determinant you need the cofactors, since det(A) = Sum_{i=1}^{n} (A_ij*C_ij) [5] So, compute det(A), keep the cofactors, transpose (Eq. [3]) - done. Use normal integer arithmetic and take into account that over Z2 all odd numbers are 1 and all even numbers are 0 (i.e., get the integer result modulo 2). If you can overload + and * operators for type Z2 you can make it pretty simple. And efficient, if you implement addition as XOR and multiplication - as AND. If you find, e.g., a decent templatized C++ inversion routine maybe you can use it as is after overloading + and *? Hope it helps, -- Oleg Goldshmidt | pub at goldshmidt.org From matan at svgalib.org Sun Aug 9 13:02:47 2015 From: matan at svgalib.org (Matan Ziv-Av) Date: Sun, 9 Aug 2015 13:02:47 +0300 (IDT) Subject: Matrix inversion tool In-Reply-To: <87614pm394.fsf@goldshmidt.org> References: <55C63D0A.7000902@shemesh.biz> <1439056571.10934.54.camel@zak.co.il> <87614pm394.fsf@goldshmidt.org> Message-ID: On Sun, 9 Aug 2015, Oleg Goldshmidt wrote: > Matan Ziv-Av writes: > >> On Sat, 8 Aug 2015, Omer Zak wrote: >> >>> What happens if you use the regular matrix inversion tool which works >>> on real numbers? (after rejecting, as singular over Z2, matrices >>> whose determinant modulo 2 is different from 1) >> >> This will work if the inverse only has integer entries. It may also >> work if your algorithm returns rational numbers, but will not work >> with floating point numbers (there are no reals in a computer). You >> cannot faithfully convert 1.1234 to an element of Z_2. > > The original matrix contains zeros and ones, and you never need to > divide, just multiply and add modulo 2. Please read what you reply to. The suggestion was to treat the matrix as a real matrix, so the calculations will not be modulo 2. If you take the, for example, matrix [ [ 0, 0, 1, 1 ], [ 0, 1, 0, 1 ], [ 1, 0, 0, 1 ], [ 1, 1, 1, 0 ] ] The inverse, in floating point numbers, will be something like [ [ -0.33333333333333331, -0.33333333333333331, 0.66666666666666663, 0.33333333333333331 ], [ -0.33333333333333331, 0.66666666666666663, -0.33333333333333331, 0.33333333333333331 ], [ 0.66666666666666663, -0.33333333333333331, -0.33333333333333331, 0.33333333333333331 ], [ 0.33333333333333331, 0.33333333333333331, 0.33333333333333331, -0.33333333333333331 ] ] And you have no idea if 0.66666666666666663 should be 0 or 1 mod 2. If you use rational numbers you get [ [ -1/3, -1/3, 2/3, 1/3 ], [ -1/3, 2/3, -1/3, 1/3 ], [ 2/3, -1/3, -1/3, 1/3 ], [ 1/3, 1/3, 1/3, -1/3 ] ] Which is easy to convert (you ignore the denominators, which will all be odd for an invertible matrix). -- Matan. From matan at svgalib.org Sun Aug 9 13:29:35 2015 From: matan at svgalib.org (Matan Ziv-Av) Date: Sun, 9 Aug 2015 13:29:35 +0300 (IDT) Subject: Matrix inversion tool In-Reply-To: <87bnehm3fk.fsf@goldshmidt.org> References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> Message-ID: On Sun, 9 Aug 2015, Oleg Goldshmidt wrote: > Shachar Shemesh writes: > >> Hi all, >> >> I'm looking for a tool/code to invert a matrix. So far, this sounds >> trivial. I have one special requirement. I did not think it was too >> special, except I could not find anywhere that supplied it. >> >> I want the matrix to be over a different field (i.e. - not the real >> numbers). In my particular case, I want it to be over Z2 (zero and >> one). > > If you don't find ready code writing your own should be pretty > simple for Z2 - fields don't get any simpler. ;-) > > In general, if you have a matrix A, then its inverse is given by > > A^(-1) = adj(A)/det(A) [1] > > The determinant of any matrix over Z2 is either 0 or 1. If it is 0 the > matrix is not invertible. If it is 1, then the inverse matrix is the > same as the adjugate of the original matrix. > > Now, the adjugate is the transpose of the cofactor matrix, which is > > C_ij = (-1)^(i+j)*M_ij [2] > > where M_ij is A's (ij)-minor, i.e., the determinant of the matrix > derived by deleting the i-th row and the j-th column of A. Thus > > adj(A) = C_ji = (-1)^(i+j)*M_ji [3] > > Thus, from equations 1 and 3 we have > > {A^(-1)}_ij = (-1)^(i+j)*M_ji [4] > > There are probably library routines to compute minors and cofactors. If > you don't find any, then, given that all you have is zeroes and ones, it > should be easy and safe to write your own. Note that you need to compute > the determinant to determine whether or not your matrix is invertible, > and to compute the determinant you need the cofactors, since > > det(A) = Sum_{i=1}^{n} (A_ij*C_ij) [5] > > So, compute det(A), keep the cofactors, transpose (Eq. [3]) - done. Use The last line is wrong. The previous are mathematically correct, but computationally wrong. When we compute det(A) using [5], we only calculate the minors for the first line, so we only know the minors for the first line, we still need to calculate them for the rest of the lines. The calculation as suggested above will run in O(n!) steps, this will obviously stop working for matrices of size 15x15 or there about. The naive algorithm, taught to any engineering or science student in the first linear algebra course, is Gauss elimination (also known as LU decomposition in this context). It runs in O(n^3) steps. Note that this algorithm is used very similarly for both inverting a matrix and calculating the determinant. So even if we replace step [5] in your suggestion, with Gaussian elimination, we still get O(n^5), when our code actually has all necessary components for running in O(n^3) instead. This is why when teaching about the classical adjoint forumula for inverse ([1]), or equivalently, the Cramer's rule for solving linear equations, it is very important to impress upon the students that while it appears "simpler" than Gauss elimination, it is in fact a lot less efficient. As a side note, the best current algorithm runs in O(n^2.3727) (at least according to wikipedia), so if you care a lot about performance, you can still save some computation time by doing more research. -- Matan. From shachar at shemesh.biz Sun Aug 9 19:50:26 2015 From: shachar at shemesh.biz (Shachar Shemesh) Date: Sun, 09 Aug 2015 19:50:26 +0300 Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> Message-ID: On 09/08/2015 13:29, Matan Ziv-Av wrote: > On Sun, 9 Aug 2015, Oleg Goldshmidt wrote: > Shachar Shemesh writes: Hi all, I'm looking for a tool/code to invert a matrix. So far, this sounds trivial. I have one special requirement. I did not think it was too special, except I could not find anywhere that supplied it. I want the matrix to be over a different field (i.e. - not the real numbers). In my particular case, I want it to be over Z2 (zero and one). If you don't find ready code writing your own should be pretty simple for Z2 - fields don't get any simpler. ;-) In general, if you have a matrix A, then its inverse is given by A^(-1) = adj(A)/det(A) [1] The determinant of any matrix over Z2 is either 0 or 1. If it is 0 the matrix is not invertible. If it is 1, then the inverse matrix is the same as the adjugate of the original matrix. Now, the adjugate is the transpose of the cofactor matrix, which is C_ij = (-1)^(i+j)*M_ij [2] where M_ij is A's (ij)-minor, i.e., the determinant of the matrix derived by deleting the i-th row and the j-th column of A. Thus adj(A) = C_ji = (-1)^(i+j)*M_ji [3] Thus, from equations 1 and 3 we have {A^(-1)}_ij = (-1)^(i+j)*M_ji [4] There are probably library routines to compute minors and cofactors. If you don't find any, then, given that all you have is zeroes and ones, it should be easy and safe to write your own. Note that you need to compute the determinant to determine whether or not your matrix is invertible, and to compute the determinant you need the cofactors, since det(A) = Sum_{i=1}^{n} (A_ij*C_ij) [5] So, compute det(A), keep the cofactors, transpose (Eq. [3]) - done. Use The last line is wrong. The previous are mathematically correct, but computationally wrong. When we compute det(A) using [5], we only calculate the minors for the first line, so we only know the minors for the first line, we still need to calculate them for the rest of the lines. The calculation as suggested above will run in O(n!) steps, this will obviously stop working for matrices of size 15x15 or there about. I should point out that the SIMPLIFIED matrix I was using to prove to myself that the idea has merit is 20x20 (inversed using state of the art in the vi text editing front). The code I'll actually have to run will be 273x273. The Matrix itself is sparsish, so it's really not _that_ bad. Still, to prove things are working I'll have to (probably pre-calculate) several thousand such matrices. In fact, one of my problems in using tools such as Mathematica and friends is that I don't want to hand code the Matrix to be inversed. Shachar -------------- next part -------------- An HTML attachment was scrubbed... URL: From matan at svgalib.org Sun Aug 9 20:15:02 2015 From: matan at svgalib.org (Matan Ziv-Av) Date: Sun, 9 Aug 2015 20:15:02 +0300 (IDT) Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> Message-ID: On Sun, 9 Aug 2015, Shachar Shemesh wrote: > I should point out that the simplified matrix I was using to prove to myself that the idea has merit is 20x20 (inversed using state of the art in the vi text editing front). The code > I'll actually have to run will be 273x273. The Matrix itself is sparsish, so it's really not that bad. Still, to prove things are working I'll have to (probably pre-calculate) several > thousand such matrices. > > In fact, one of my problems in using tools such as Mathematica and friends is that I don't want to hand code the Matrix to be inversed. GAP (and similarly SAGE, maxima) can be scripted. You do not give your exact setup, but if you can write the matrices to a text file in any format, you can read it in GAP. If you can output to the same format as I posted, you do not even need to write any program in GAP to read it. -- Matan. From matan at svgalib.org Mon Aug 10 00:57:13 2015 From: matan at svgalib.org (Matan Ziv-Av) Date: Mon, 10 Aug 2015 00:57:13 +0300 (IDT) Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> Message-ID: On Sun, 9 Aug 2015, Shachar Shemesh wrote: > I should point out that the simplified matrix I was using to prove to > myself that the idea has merit is 20x20 (inversed using state of the > art in the vi text editing front). The code I'll actually have to run > will be 273x273. The Matrix itself is sparsish, so it's really not > that bad. Still, to prove things are working I'll have to (probably > pre-calculate) several thousand such matrices. Actually, for this specific problem, implementing Gaussian elimination is much better than looking for a library or a tool, and interfacing with it. Untested C-Pseudo code: Input: A,B: nxn matrices, B=I_n Output: det(A), B=A^{-1} if A det=1 for(i=0;i References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> Message-ID: On Aug 9, 2015 20:15, "Matan Ziv-Av" wrote: > > On Sun, 9 Aug 2015, Shachar Shemesh wrote: > >> I should point out that the simplified matrix I was using to prove to myself that the idea has merit is 20x20 (inversed using state of the art in the vi text editing front). The code >> I'll actually have to run will be 273x273. The Matrix itself is sparsish, so it's really not that bad. Still, to prove things are working I'll have to (probably pre-calculate) several >> thousand such matrices. >> >> In fact, one of my problems in using tools such as Mathematica and friends is that I don't want to hand code the Matrix to be inversed. > > > GAP (and similarly SAGE, maxima) can be scripted. > > You do not give your exact setup, but if you can write the matrices to a text file in any format, you can read it in GAP. If you can output to the same format as I posted, you do not even need to write any program in GAP to read it. > Not being connected to the field of applied mathematics can only recommend solutions observed. There is vast amount of code already written for LinAlg. in python you can use numpy. In C(++) any BLAS library have all you need. OpenBLAS seems more effective. If yo need to manipulate matrices "en masse" strongly suggested to use GPU. Both OpenCL and CUDA are good. Best Regards, Evgeniy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pub at goldshmidt.org Mon Aug 10 09:11:05 2015 From: pub at goldshmidt.org (Oleg Goldshmidt) Date: Mon, 10 Aug 2015 09:11:05 +0300 Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <1439056571.10934.54.camel@zak.co.il> <87614pm394.fsf@goldshmidt.org> Message-ID: <87wpx3lll2.fsf@goldshmidt.org> Matan Ziv-Av writes: > Please read what you reply to. I assure you I read and understood Omer's and your posts. If you go back to my reply you will surely realize that at no point I contradicted you. I just pointed out that you didn't need to divide (by det(A)==1), which would lead to the problem you correctly point out. > The suggestion was to treat the matrix as a real matrix, so the > calculations will not be modulo 2. > > If you take the, for example, matrix > [ [ 0, 0, 1, 1 ], > [ 0, 1, 0, 1 ], > [ 1, 0, 0, 1 ], > [ 1, 1, 1, 0 ] ] > > The inverse, in floating point numbers, will be something like > [ [ -0.33333333333333331, -0.33333333333333331, 0.66666666666666663, > 0.33333333333333331 ], > [ -0.33333333333333331, 0.66666666666666663, -0.33333333333333331, > 0.33333333333333331 ], > [ 0.66666666666666663, -0.33333333333333331, -0.33333333333333331, > 0.33333333333333331 ], > [ 0.33333333333333331, 0.33333333333333331, 0.33333333333333331, > -0.33333333333333331 ] ] To get that result you divided (by the determinant). All I said was you did not need to. Apply mod 2 to all the (integer[*]) addition results, and your determinant will be 1 (over Z2) and not 3. Friends again? ;-) [*] I strongly suspect you read too much into Omer's use of the word "real" (also used in the OP). -- Oleg Goldshmidt | pub at goldshmidt.org From pub at goldshmidt.org Mon Aug 10 09:23:59 2015 From: pub at goldshmidt.org (Oleg Goldshmidt) Date: Mon, 10 Aug 2015 09:23:59 +0300 Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> Message-ID: <87vbcnlkzk.fsf@goldshmidt.org> Matan Ziv-Av writes: > The last line is wrong. You are right. > The naive algorithm, taught to any engineering or science student in > the first linear algebra course, is Gauss elimination (also known as > LU decomposition in this context). It runs in O(n^3) steps. > > Note that this algorithm is used very similarly for both inverting a > matrix and calculating the determinant. So even if we replace step [5] > in your suggestion, with Gaussian elimination, we still get O(n^5), > when our code actually has all necessary components for running in > O(n^3) instead. A general comment. Asymptotic complexity has its uses but is very rarely relevant in practice. One would probaly need a serious literature search just to find out on what scale asymptotic complexity becomes relevant for a given type of problem, and I will not be surprised if such a search gives no generic result, or no practically useful result. In practice, people keep solving problems that would require Hubble times asymptotically. There was no scale (or speed) requirement mentioned in the OP. Later, Shachar mentioned 273x273 as a target. [He also said it was likely to be sparse, so I wouldn't get stuck on O() estimates.] I admit I have not done much matrix manipulation for quite a while, but 273x273 seems quite large for a regular computer. It is not clear to me (I'll appreciate a comment just for curiousity's sake as well as for education) that a straightforward algorithm will be practical. Maybe one needs to think of a highly paralellized algo on such scales. A CUDA (GPGPU, MIC, whatever) computer may help, may or may not require custom implementation, and may or may not make any differences in asymptotic complexity go away on the 300x300 scale. On top of that, it is not clear to me how much the fact that + is XOR and * is AND over Z2 may help (with or withut CUDA or similar). -- Oleg Goldshmidt | pub at goldshmidt.org From shachar at shemesh.biz Mon Aug 10 16:15:55 2015 From: shachar at shemesh.biz (Shachar Shemesh) Date: Mon, 10 Aug 2015 16:15:55 +0300 Subject: Matrix inversion tool In-Reply-To: <87vbcnlkzk.fsf@goldshmidt.org> References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> <87vbcnlkzk.fsf@goldshmidt.org> Message-ID: <55C8A40B.70107@shemesh.biz> On 10/08/15 09:23, Oleg Goldshmidt wrote: > A general comment. Asymptotic complexity has its uses but is very rarely > relevant in practice. One would probaly need a serious literature search > just to find out on what scale asymptotic complexity becomes relevant > for a given type of problem, and I will not be surprised if such a > search gives no generic result, or no practically useful result. In > practice, people keep solving problems that would require Hubble times > asymptotically. > > There was no scale (or speed) requirement mentioned in the OP. Later, > Shachar mentioned 273x273 as a target. [He also said it was likely to be > sparse, so I wouldn't get stuck on O() estimates.] I admit I have not > done much matrix manipulation for quite a while, but 273x273 seems quite > large for a regular computer. 74,000 cells hardly seem beyond the realm of recent[1] computer's power to crunch. Even assuming straight on n^3, this means handling 20 million cells. Do it in a non-compiled language, and it might take you a full second (that's 100 cycles just to perform a single XOR). > It is not clear to me (I'll appreciate a > comment just for curiousity's sake as well as for education) that a > straightforward algorithm will be practical. As I've said before, I've inversed, using Gaussian elimination, a 20x20 (400 cells) matrix using nothing more than vi. I'll add that the bigger the matrix, the sparser it is (all lines except 12, regardless of matrix size, are already diagonal). Even had that not been the case, however, I fail to see how these are matrix sizes beyond the reach of normal computer's capabilities. Shachar 1 - say, state of the art circa 2000. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matan at svgalib.org Mon Aug 10 17:32:56 2015 From: matan at svgalib.org (Matan Ziv-Av) Date: Mon, 10 Aug 2015 17:32:56 +0300 (IDT) Subject: Matrix inversion tool In-Reply-To: <87wpx3lll2.fsf@goldshmidt.org> References: <55C63D0A.7000902@shemesh.biz> <1439056571.10934.54.camel@zak.co.il> <87614pm394.fsf@goldshmidt.org> <87wpx3lll2.fsf@goldshmidt.org> Message-ID: On Mon, 10 Aug 2015, Oleg Goldshmidt wrote: > I assure you I read and understood Omer's and your posts. If you go back > to my reply you will surely realize that at no point I contradicted > you. I just pointed out that you didn't need to divide (by det(A)==1), > which would lead to the problem you correctly point out. Here's what Omer actually asked: > What happens if you use the regular matrix inversion tool which works > on real numbers? And I explained that a regular matrix inversion tool will return a result that cannot be converted back to Z_2. If you are using another tool, you do not have the option to "not divide by the determinant", since that too returns the actual inverse matrix after the "division by the determinant" (though, it never actually divides by the determinant). > Friends again? ;-) Obviously, I have no personal problems with you. I just don't like incorrect information (about a subject I really care about) available, since it has a way of returning very unexpectedly. To show that there is no animosity, I'll answer your other questions: > 273x273 seems quite > large for a regular computer. It is not clear to me (I'll appreciate a > comment just for curiousity's sake as well as for education) that a > straightforward algorithm will be practical. At the end of this message is a python program with simple implementations of both algorithms (Gauss eliminiation and recursive). Run them for sizes 10 to 16 consecuitively to see how the difference between exponential and polynomial is very significant in this case. Then run for 273 (or 2730, for that matter), to see that with the right algorithms, 273x273 is very small for a computer. If you save it as x.py, run as ./x.py n k s Where n is the number of rows, k is the expected number of non-zeros in a row, and the optional s is a seed for the pseudo-random number generator. The output is the determinant by GE and the time it takes, and then the determinant by the recursive method and then the time it takes. The optimizations you suggested are possible (for both algorithms), but they are all just scaling the time needed. Even if you speed up by a factor of 10^9, the recursive algorithm will just not work. The sparsity of the matrix does not help much. If there are (on average) k non-zero entries per line, you get O(k^n) instead of O(n!). > [*] I strongly suspect you read too much into Omer's use of the word > "real" (also used in the OP). Actually he said real numbers. How can you read it in any other way? especially in this context. > > -- Matan. #!/usr/bin/python import random import sys import time import copy def mat(n,p): return [[int(random.random()=4: random.seed(int(sys.argv[3])) a1=mat(n,p/n) a2=copy.deepcopy(a1) b=id(n) s=time.clock() print "d1=",ge(a1,b) e=time.clock() print e-s s=time.clock() print "d2=",recdet(a2) e=time.clock() print e-s From w1 at zak.co.il Mon Aug 10 20:56:52 2015 From: w1 at zak.co.il (Omer Zak) Date: Mon, 10 Aug 2015 20:56:52 +0300 Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> Message-ID: <1439229412.10934.108.camel@zak.co.il> All those discussions about inverting matrices over Z2 make me curious to know what kind of problems can be solved by inverting such matrices. I suppose that the actual problem, with which Shachar is struggling, is proprietary information. However, is it possible to indicate the kind of problems which can be attacked by inverting a matrix over Z2? --- Omer On Sun, 2015-08-09 at 19:50 +0300, Shachar Shemesh wrote: > On 09/08/2015 13:29, Matan Ziv-Av wrote: > > > On Sun, 9 Aug 2015, Oleg Goldshmidt wrote: > > > Shachar Shemesh writes: > > > > Hi all, I'm looking for a tool/code to invert a matrix. So far, > > > > this sounds trivial. I have one special requirement. I did not > > > > think it was too special, except I could not find anywhere that > > > > supplied it. I want the matrix to be over a different field > > > > (i.e. - not the real numbers). In my particular case, I want it > > > > to be over Z2 (zero and one). [... snipped ...] -- The brain does not use addresses. www.werner-seyfried.com My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html From shlomif at gmail.com Mon Aug 10 21:48:53 2015 From: shlomif at gmail.com (Shlomi Fish) Date: Mon, 10 Aug 2015 21:48:53 +0300 Subject: Matrix inversion tool In-Reply-To: <1439229412.10934.108.camel@zak.co.il> References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> <1439229412.10934.108.camel@zak.co.il> Message-ID: Hi Omer, On Mon, Aug 10, 2015 at 8:56 PM, Omer Zak wrote: > All those discussions about inverting matrices over Z2 make me curious > to know what kind of problems can be solved by inverting such matrices. > > I suppose that the actual problem, with which Shachar is struggling, is > proprietary information. However, is it possible to indicate the kind of > problems which can be attacked by inverting a matrix over Z2? > Some examples for such problems are http://www.shlomifish.org/MathVentures/toggle_squares.html and the similar puzzle of http://www.logicgamesonline.com/lightsout/daily.php . Hope it helps! Regards, -- Shlomi -- ------------------------------------------ Shlomi Fish http://www.shlomifish.org/ Chuck Norris helps the gods that help themselves. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From shachar at shemesh.biz Mon Aug 10 22:56:05 2015 From: shachar at shemesh.biz (Shachar Shemesh) Date: Mon, 10 Aug 2015 22:56:05 +0300 Subject: Matrix inversion tool In-Reply-To: <1439229412.10934.108.camel@zak.co.il> References: <55C63D0A.7000902@shemesh.biz> <87bnehm3fk.fsf@goldshmidt.org> <1439229412.10934.108.camel@zak.co.il> Message-ID: <55C901D5.5070809@shemesh.biz> On 10/08/15 20:56, Omer Zak wrote: > All those discussions about inverting matrices over Z2 make me curious > to know what kind of problems can be solved by inverting such matrices. > > I suppose that the actual problem, with which Shachar is struggling, is > proprietary information. However, is it possible to indicate the kind of > problems which can be attacked by inverting a matrix over Z2? I think I'm not letting on too much by giving the broad area. When calculating RAID-6 parity, one of the common methods is called "erasure coding". You have n data blocks in a stripe which you need to write to n+p actual disks, such that for any number of failures up to p, you can recover the original n data blocks. One of the ways to do that is by employing a family of algorithms called "Erasure codes", the most famous of which is Reed Solomon. The basic idea is that you write a matrix which is n over n+p, and you multiply the n data blocks by that matrix, resulting in n+p data blocks, which you then write to your disks. When p disks failed, you erase from your matrix the lines that correspond to the disks that failed, and invert the resulting n*n matrix. Multiplying the n disks you do have by that matrix result in the origin n data blocks. The obvious first requirement from your erasure code matrix, therefor, is that it be inversible after deleting any combination of p lines from it. This is the property I'm trying to verify in my project at hand. The usual erasure code matrix is calculated over "ze ozer finite field", i.e. Galois field. Frankly, I don't feel I understand Galois field, and in particular, the way it is used in this particular case, enough to explain why. One of the properties that make it attractive, however, is that addition over GF is XOR. As a result, almost all RAID erasure code matrices have their first n+1 lines look like this: 1 0 0 0 .... 0 1 0 0 .... 0 0 1 0 .... 0 0 0 1 .... ........1 0 0 ........0 1 0 ........0 0 1 1 1 1 ... 1 1 The practical upshot of the above is that the first n disks receive the same data they would have received without applying erasure coding (i.e. - if they are available, they can be read directly without going through the matrix, and more importantly, without reading the rest of the disks in the stripe). Furthermore, the first protection disk contains the XOR of all data disks in the stripe. In other words, if we lost only one disk, we can employ RAID-5 as usual. A side effect of the above is that the matrix is extremely sparse. Almost all lines (all but the protection disks) contain just one non-zero number, already on the diagonal. Using Z2 instead of GF gives certain performance benefits. It does have the cost of making the matrix much much much larger (hence the somewhat insane numbers quoted in my original mail. I am not really building a stripe of 272+12). Since the matrix is still extremely sparse, however, this does not cost the full n^3 quoted by Matan. Look up "evenodd" for an example of an algorithm that is based on XOR rather than GF. It is not phrased in matrix form, but that is just a convenience. Hope this gives enough of an insight to quelch your curiosity :-) Shachar -------------- next part -------------- An HTML attachment was scrubbed... URL: From shachar at shemesh.biz Mon Aug 10 23:02:02 2015 From: shachar at shemesh.biz (Shachar Shemesh) Date: Mon, 10 Aug 2015 23:02:02 +0300 Subject: Matrix inversion tool In-Reply-To: References: <55C63D0A.7000902@shemesh.biz> <1439056571.10934.54.camel@zak.co.il> <87614pm394.fsf@goldshmidt.org> <87wpx3lll2.fsf@goldshmidt.org> Message-ID: <55C9033A.60908@shemesh.biz> On 10/08/15 17:32, Matan Ziv-Av wrote: > > At the end of this message is a python program with simple > implementations of both algorithms (Gauss eliminiation and recursive). > Run them for sizes 10 to 16 consecuitively to see how the difference > between exponential and polynomial is very significant in this case. 1. Ask for help finding tools on mailing list 2. Get Oleg to question the viability of the suggested solution 3. Matan writes the tool you need for you 4. ???? 5. Profit > The optimizations you suggested are possible (for both algorithms), > but they are all just scaling the time needed. Even if you speed up by > a factor of 10^9, the recursive algorithm will just not work. I have not delved too deeply into the "det" calculation, but wouldn't caching of sub-results cause an actual decrease in the complexity of the run? I've had a program I wrote once (for solving Nonograms (https://en.wikipedia.org/wiki/Nonogram)) where adding caching of intermediate results caused an complexity reduction. A riddle previously solved in ~40 minutes was, after introducing caching, solved in 150 millisconds!!! I even considered doing a Haifux lecture titled "Complexity; why programmers ignore it, and why it still matters". Shachar -------------- next part -------------- An HTML attachment was scrubbed... URL: From shlomif at gmail.com Wed Aug 12 21:22:40 2015 From: shlomif at gmail.com (Shlomi Fish) Date: Wed, 12 Aug 2015 21:22:40 +0300 Subject: New Page on My Site about Computer Graphics Applications Message-ID: Hi all! I created a new web page listing some prominent Computer Graphics Applications here: http://www.shlomifish.org/open-source/resources/graphics-programs/ The licence is CC-by. Suggestions are welcome. Regards, Shlomi Fish -- ------------------------------------------ Shlomi Fish http://www.shlomifish.org/ Chuck Norris helps the gods that help themselves. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From slitt at troubleshooters.com Thu Aug 13 08:20:22 2015 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 13 Aug 2015 01:20:22 -0400 Subject: New Page on My Site about Computer Graphics Applications In-Reply-To: References: Message-ID: <20150813012022.3bd94894@mydesq2.domain.cxm> On Wed, 12 Aug 2015 21:22:40 +0300 Shlomi Fish wrote: > Hi all! > > I created a new web page listing some prominent Computer Graphics > Applications > here: > > http://www.shlomifish.org/open-source/resources/graphics-programs/ Very nice! I'm going to try some of those. > > The licence is CC-by. Suggestions are welcome. I'd suggest adding dia. Dia is a poor man's Visio, and very handy for diagrams. You might need to add a new category for it. http://dia-installer.de/ Thanks for the great page. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust From shlomif at gmail.com Thu Aug 13 10:30:43 2015 From: shlomif at gmail.com (Shlomi Fish) Date: Thu, 13 Aug 2015 10:30:43 +0300 Subject: New Page on My Site about Computer Graphics Applications In-Reply-To: <20150813012022.3bd94894@mydesq2.domain.cxm> References: <20150813012022.3bd94894@mydesq2.domain.cxm> Message-ID: Hi Steve, On Thu, Aug 13, 2015 at 8:20 AM, Steve Litt wrote: > On Wed, 12 Aug 2015 21:22:40 +0300 > Shlomi Fish wrote: > > > Hi all! > > > > I created a new web page listing some prominent Computer Graphics > > Applications > > here: > > > > http://www.shlomifish.org/open-source/resources/graphics-programs/ > > Very nice! I'm going to try some of those. > > Thanks! > > > > The licence is CC-by. Suggestions are welcome. > > I'd suggest adding dia. Dia is a poor man's Visio, and very handy for > diagrams. You might need to add a new category for it. > > http://dia-installer.de/ > > You are right - I need to add Dia as well as Visio and possibly https://en.wikipedia.org/wiki/Calligra_Flow as well. I recall intending to add them , but that I forgot about it. Thanks for reminding me. > Thanks for the great page. > > You're welcome. Regards, -- Shlomi Fish > SteveT > > Steve Litt > August 2015 featured book: Troubleshooting: Just the Facts > http://www.troubleshooters.com/tjust > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > -- ------------------------------------------ Shlomi Fish http://www.shlomifish.org/ Chuck Norris helps the gods that help themselves. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From slitt at troubleshooters.com Thu Aug 13 19:14:57 2015 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 13 Aug 2015 12:14:57 -0400 Subject: New Page on My Site about Computer Graphics Applications In-Reply-To: References: <20150813012022.3bd94894@mydesq2.domain.cxm> Message-ID: <20150813121457.7b046b1a@mydesq2.domain.cxm> On Thu, 13 Aug 2015 10:30:43 +0300 Shlomi Fish wrote: > You are right - I need to add Dia as well as Visio and possibly > https://en.wikipedia.org/wiki/Calligra_Flow as well. I don't think Visio is Free Software. Search "visio for linux", *without the quotes*, and you'll get a lot of great info mixed with recommendations of commercial software. I found this page particularly interesting: http://askubuntu.com/questions/37632/is-there-an-alternative-to-microsoft-office-visio By the way, on the preceding link, search the word "connector" to see how to make Inkscape behave as a real diagramming tool. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust From shlomif at gmail.com Thu Aug 13 20:24:12 2015 From: shlomif at gmail.com (Shlomi Fish) Date: Thu, 13 Aug 2015 20:24:12 +0300 Subject: New Page on My Site about Computer Graphics Applications In-Reply-To: <20150813121457.7b046b1a@mydesq2.domain.cxm> References: <20150813012022.3bd94894@mydesq2.domain.cxm> <20150813121457.7b046b1a@mydesq2.domain.cxm> Message-ID: Hi Steve, On Thu, Aug 13, 2015 at 7:14 PM, Steve Litt wrote: > On Thu, 13 Aug 2015 10:30:43 +0300 > Shlomi Fish wrote: > > > > You are right - I need to add Dia as well as Visio and possibly > > https://en.wikipedia.org/wiki/Calligra_Flow as well. > > I don't think Visio is Free Software. Visio is indeed not open source, but I don't link to open source software in my resources (directories) pages exclusively, though I have a preference for listing such applications. Search "visio for linux", > *without the quotes*, and you'll get a lot of great info mixed with > recommendations of commercial software. > > I found this page particularly interesting: > > > http://askubuntu.com/questions/37632/is-there-an-alternative-to-microsoft-office-visio > > By the way, on the preceding link, search the word "connector" to see > how to make Inkscape behave as a real diagramming tool. > > Thanks! I'll check it out. Regards, ? Shlomi Fish > > SteveT > > Steve Litt > August 2015 featured book: Troubleshooting: Just the Facts > http://www.troubleshooters.com/tjust > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > -- ------------------------------------------ Shlomi Fish http://www.shlomifish.org/ Chuck Norris helps the gods that help themselves. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.s at daniel.shahaf.name Fri Aug 21 04:24:27 2015 From: d.s at daniel.shahaf.name (Daniel Shahaf) Date: Fri, 21 Aug 2015 01:24:27 +0000 Subject: [OT] driver's license exam app? Message-ID: <20150821012427.GD1840@tarsus.local2> I've got here a desktop app that quizzes the user with questions from ???? ???????? (the one people take when they learn driving). It's basically a self-test/study app, using the questionset from the Ministry's web site. (They publish questionsets in six languages, I only tried the Hebrew set.) By "app" I mean a python script that prints questions to stdout, reads answers from stdin, and displays images by invoking display(1) [a minimal image viewer from imagemagick]. It doesn't have a GUI (beyond the image displayer) since my target audience didn't need one. I can't release it as-is because $LEGAL_REASONS, but I could clean it up to make it releaseable. Before I spend too much time on that, is that something anybody would be interested in? [feel free to reply offlist] Cheers, Daniel P.S. It wasn't fun to get Hebrew to print correctly on all terminals: there are differently-behaving terminals that use the same value of $TERM (undermining terminfo-based solutions). I ended up using $WINDOWID to get the terminal emulator's argv[0], and hardcoding exceptions based on that. That's so 1990... From w1 at zak.co.il Fri Aug 21 07:51:51 2015 From: w1 at zak.co.il (Omer Zak) Date: Fri, 21 Aug 2015 07:51:51 +0300 Subject: [OT] driver's license exam app? In-Reply-To: <20150821012427.GD1840@tarsus.local2> References: <20150821012427.GD1840@tarsus.local2> Message-ID: <1440132711.4223.196.camel@zak.co.il> Hello Daniel, Three years ago, I developed an Android application which does the same thing [1]. I hoped to get more volunteers to polish and improve it, however there was very little interest in it. Nevertheless, it was a good way to tell the world that yes, I am an Android developer. I got other projects thanks to it (such as [2]). So I suggest that you release your application but without expecting anything from it beyond being a showcase of your software development skills. While you are it, I strongly urge you to split off the part of the code, which deals with Hebrew terminals, and make it a library that people can take and use in their own projects. Maybe you'll be able to contribute to terminfo/ncurses projects. --- Omer [1] https://github.com/Hamakor/teuria [2] https://play.google.com/store/apps/details?id=com.heliconbooks.epub.epubreader On Fri, 2015-08-21 at 01:24 +0000, Daniel Shahaf wrote: > I've got here a desktop app that quizzes the user with questions from basically a self-test/study app, using the questionset from the > Ministry's web site. (They publish questionsets in six languages, > I only tried the Hebrew set.) > > By "app" I mean a python script that prints questions to stdout, reads > answers from stdin, and displays images by invoking display(1) [a > minimal image viewer from imagemagick]. It doesn't have a GUI (beyond > the image displayer) since my target audience didn't need one. > > I can't release it as-is because $LEGAL_REASONS, but I could clean it up > to make it releaseable. Before I spend too much time on that, is that > something anybody would be interested in? > > [feel free to reply offlist] > > Cheers, > > Daniel > > P.S. It wasn't fun to get Hebrew to print correctly on all terminals: > there are differently-behaving terminals that use the same value of > $TERM (undermining terminfo-based solutions). I ended up using > $WINDOWID to get the terminal emulator's argv[0], and hardcoding > exceptions based on that. That's so 1990... -- QA People of Curse. My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html From dov.grobgeld at gmail.com Sat Aug 22 22:53:05 2015 From: dov.grobgeld at gmail.com (Dov Grobgeld) Date: Sat, 22 Aug 2015 22:53:05 +0300 Subject: Major rendering bug affecting Hebrew in pango has been fixed Message-ID: This might be interesting to someone on the list. A major bug affecting rendering of Hebrew with justification in pango has been fixed after more than 8 years. See the following animated gif showing the rendering before and after the latest pango updates. https://bug753772.bugzilla-attachments.gnome.org/attachment.cgi?id=309871 Though the animated gif shows Hebrew with nikud, the problem exists just the same without nikud. My only contribution was reporting the bug and giving feedback to Behdad who fixed it. Regards, Dov -------------- next part -------------- An HTML attachment was scrubbed... URL: From amos.shapira at gmail.com Sun Aug 23 04:05:08 2015 From: amos.shapira at gmail.com (Amos Shapira) Date: Sun, 23 Aug 2015 11:05:08 +1000 Subject: Major rendering bug affecting Hebrew in pango has been fixed In-Reply-To: References: Message-ID: Is this the same code used in Android? Just today I read a Hebrew article in Pocket (https://getpocket.com/) on my Nexus 5 and was reminded that it still justifies the "mobilized" version to the left. On 23 August 2015 at 05:53, Dov Grobgeld wrote: > This might be interesting to someone on the list. > > A major bug affecting rendering of Hebrew with justification in pango has > been fixed after more than 8 years. > > See the following animated gif showing the rendering before and after the > latest pango updates. > > https://bug753772.bugzilla-attachments.gnome.org/attachment.cgi?id=309871 > > Though the animated gif shows Hebrew with nikud, the problem exists just > the same without nikud. > > My only contribution was reporting the bug and giving feedback to Behdad > who fixed it. > > Regards, > Dov > > > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dov.grobgeld at gmail.com Sun Aug 23 10:07:12 2015 From: dov.grobgeld at gmail.com (Dov Grobgeld) Date: Sun, 23 Aug 2015 10:07:12 +0300 Subject: Major rendering bug affecting Hebrew in pango has been fixed In-Reply-To: References: Message-ID: No, pango isn't used in Android, so there is no connection. Pango is mainly used in gnome and gtk programs like inkscape and gimp. Dov On Sun, Aug 23, 2015 at 4:05 AM, Amos Shapira wrote: > Is this the same code used in Android? > Just today I read a Hebrew article in Pocket (https://getpocket.com/) on > my Nexus 5 and was reminded that it still justifies the "mobilized" version > to the left. > > On 23 August 2015 at 05:53, Dov Grobgeld wrote: > >> This might be interesting to someone on the list. >> >> A major bug affecting rendering of Hebrew with justification in pango has >> been fixed after more than 8 years. >> >> See the following animated gif showing the rendering before and after the >> latest pango updates. >> >> https://bug753772.bugzilla-attachments.gnome.org/attachment.cgi?id=309871 >> >> Though the animated gif shows Hebrew with nikud, the problem exists just >> the same without nikud. >> >> My only contribution was reporting the bug and giving feedback to Behdad >> who fixed it. >> >> Regards, >> Dov >> >> >> >> >> _______________________________________________ >> Linux-il mailing list >> Linux-il at cs.huji.ac.il >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >> >> > > > -- > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.s at daniel.shahaf.name Sun Aug 23 11:18:56 2015 From: d.s at daniel.shahaf.name (Daniel Shahaf) Date: Sun, 23 Aug 2015 08:18:56 +0000 Subject: [OT] driver's license exam app? In-Reply-To: <1440132711.4223.196.camel@zak.co.il> References: <20150821012427.GD1840@tarsus.local2> <1440132711.4223.196.camel@zak.co.il> Message-ID: <20150823081856.GG5472@tarsus.local2> Omer Zak wrote on Fri, Aug 21, 2015 at 07:51:51 +0300: > Hello Daniel, > Three years ago, I developed an Android application which does the same > thing [1]. > > I hoped to get more volunteers to polish and improve it, however there > was very little interest in it. Haven't seen it, but in my case I needed a desktop app, not a smartphone one. > Nevertheless, it was a good way to tell the world that yes, I am an > Android developer. I got other projects thanks to it (such as [2]). > > So I suggest that you release your application but without expecting > anything from it beyond being a showcase of your software development > skills. That's good advice. However, I don't think the code showcases anything: it's a chunk of glue code that any first-year student could write. It saves time but is not enlightening to read. The only value I see in it is that it might save someone else the need to redo the work I have done. > While you are it, I strongly urge you to split off the part of the code, > which deals with Hebrew terminals, and make it a library that people can > take and use in their own projects. Maybe you'll be able to contribute > to terminfo/ncurses projects. All I have is a chunk of code that decides whether or not to insert fribidi into the output filter chain based on whether $(ps -o comm= -p $(xprop -id $WINDOWID _NET_WM_PID | sed 's/.*= *//g')) contains "mlterm" or "konsole". While that chunk of code does solve a problem that is in scope for ncurses, I believe the approach it takes is unsuitable for inclusion therein, for several reasons: (a)?it would require ncurses to link libx11 (to get window properties); (b)?it hardcodes terminal names that behave differently ? that is a duct-tape solution, not a long-term one; (c)?it is sometimes wrong: konsole can be configured to behave either way, but the code hardcodes a specific behaviour for it. In short, what I have is a duct-tape solution, not a long-term one. That was fine for the context I wrote it in (I needed to get my app to work in today's terminal emulators without changing them), but it's not suitable for upstreaming. The proper long-term fix would be either to make terminal emulators all behave the same way with respect to directionality, or to invent a terminfo capability that terminals would set in their terminfo descriptors and ncurses could query. Cheers, Daniel > --- Omer > > > > [1] https://github.com/Hamakor/teuria > [2] > https://play.google.com/store/apps/details?id=com.heliconbooks.epub.epubreader > > > On Fri, 2015-08-21 at 01:24 +0000, Daniel Shahaf wrote: > > I've got here a desktop app that quizzes the user with questions from > > basically a self-test/study app, using the questionset from the > > Ministry's web site. (They publish questionsets in six languages, > > I only tried the Hebrew set.) > > > > By "app" I mean a python script that prints questions to stdout, reads > > answers from stdin, and displays images by invoking display(1) [a > > minimal image viewer from imagemagick]. It doesn't have a GUI (beyond > > the image displayer) since my target audience didn't need one. > > > > I can't release it as-is because $LEGAL_REASONS, but I could clean it up > > to make it releaseable. Before I spend too much time on that, is that > > something anybody would be interested in? > > > > [feel free to reply offlist] > > > > Cheers, > > > > Daniel > > > > P.S. It wasn't fun to get Hebrew to print correctly on all terminals: > > there are differently-behaving terminals that use the same value of > > $TERM (undermining terminfo-based solutions). I ended up using > > $WINDOWID to get the terminal emulator's argv[0], and hardcoding > > exceptions based on that. That's so 1990... > -- > QA People of Curse. > My own blog is at http://www.zak.co.il/tddpirate/ > > My opinions, as expressed in this E-mail message, are mine alone. > They do not represent the official policy of any organization with which > I may be affiliated in any way. > WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il From rabin at rabin.io Sun Aug 23 11:47:02 2015 From: rabin at rabin.io (Rabin Yasharzadehe) Date: Sun, 23 Aug 2015 11:47:02 +0300 Subject: [OT] driver's license exam app? In-Reply-To: <20150821012427.GD1840@tarsus.local2> References: <20150821012427.GD1840@tarsus.local2> Message-ID: On Fri, Aug 21, 2015 at 4:24 AM, Daniel Shahaf wrote: > I've got here a desktop app that quizzes the user with questions from > ???? ???????? (the one people take when they learn driving). It's > basically a self-test/study app, using the questionset from the > Ministry's web site. (They publish questionsets in six languages, > I only tried the Hebrew set.) > > By "app" I mean a python script that prints questions to stdout, reads > answers from stdin, and displays images by invoking display(1) [a > minimal image viewer from imagemagick]. It doesn't have a GUI (beyond > the image displayer) since my target audience didn't need one. > > I can't release it as-is because $LEGAL_REASONS, but I could clean it up > to make it releaseable. Before I spend too much time on that, is that > something anybody would be interested in? > > [feel free to reply offlist] > > Cheers, > > Daniel > > P.S. It wasn't fun to get Hebrew to print correctly on all terminals: > there are differently-behaving terminals that use the same value of > $TERM (undermining terminfo-based solutions). I ended up using > $WINDOWID to get the terminal emulator's argv[0], and hardcoding > exceptions based on that. That's so 1990... > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > ? I wood live to see it. -- Rabin ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From amichai at iglu.org.il Sun Aug 23 23:07:02 2015 From: amichai at iglu.org.il (Amichai Rotman) Date: Sun, 23 Aug 2015 23:07:02 +0300 Subject: [OT] Deleting Thousends of Messages in Gmail Message-ID: I know it's kinda off-topic, but I am really at a loss... I m trying to free spcace on my Google free storage, so i am sifting through very old messages to delete in my Gmail box (as far as 2005 and beyond!) I have this one label containing 12,000(!) messages. I'd like to delete all messages dated before the current year. I can't find any way to tell it to filter all messages with a certain label where the date **does not** equal current year. I hope I explained myself clearly enough. Thanks! Amichai. -------------- next part -------------- An HTML attachment was scrubbed... URL: From romovs at gmail.com Sun Aug 23 23:26:28 2015 From: romovs at gmail.com (Roman Ovseitsev) Date: Sun, 23 Aug 2015 23:26:28 +0300 Subject: [OT] Deleting Thousends of Messages in Gmail In-Reply-To: References: Message-ID: label:your_label before:2014/12/31 should do it. On Sun, Aug 23, 2015 at 11:07 PM, Amichai Rotman wrote: > I know it's kinda off-topic, but I am really at a loss... > > I m trying to free spcace on my Google free storage, so i am sifting > through very old messages to delete in my Gmail box (as far as 2005 and > beyond!) > > I have this one label containing 12,000(!) messages. I'd like to delete > all messages dated before the current year. > > I can't find any way to tell it to filter all messages with a certain > label where the date **does not** equal current year. > > I hope I explained myself clearly enough. > > Thanks! > > Amichai. > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shlomif at gmail.com Sun Aug 23 23:48:57 2015 From: shlomif at gmail.com (Shlomi Fish) Date: Sun, 23 Aug 2015 23:48:57 +0300 Subject: [OT] Deleting Thousends of Messages in Gmail In-Reply-To: References: Message-ID: Hi all, On Sun, Aug 23, 2015 at 11:26 PM, Roman Ovseitsev wrote: > label:your_label before:2014/12/31 > should do it. > > Thansk for the enlightening reply. That put aside, in general one should be able to use an API (such as IMAP) and some quick code to do that. Regards, -- Shlomi Fish > On Sun, Aug 23, 2015 at 11:07 PM, Amichai Rotman > wrote: > >> I know it's kinda off-topic, but I am really at a loss... >> >> I m trying to free spcace on my Google free storage, so i am sifting >> through very old messages to delete in my Gmail box (as far as 2005 and >> beyond!) >> >> I have this one label containing 12,000(!) messages. I'd like to delete >> all messages dated before the current year. >> >> I can't find any way to tell it to filter all messages with a certain >> label where the date **does not** equal current year. >> >> I hope I explained myself clearly enough. >> >> Thanks! >> >> Amichai. >> >> _______________________________________________ >> Linux-il mailing list >> Linux-il at cs.huji.ac.il >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >> >> > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -- ------------------------------------------ Shlomi Fish http://www.shlomifish.org/ Chuck Norris helps the gods that help themselves. Please reply to list if it's a mailing list post - http://shlom.in/reply . -------------- next part -------------- An HTML attachment was scrubbed... URL: From slitt at troubleshooters.com Mon Aug 24 04:28:30 2015 From: slitt at troubleshooters.com (Steve Litt) Date: Sun, 23 Aug 2015 21:28:30 -0400 Subject: [OT] Deleting Thousends of Messages in Gmail In-Reply-To: References: Message-ID: <20150823212830.76c487a1@mydesq2.domain.cxm> On Sun, 23 Aug 2015 23:07:02 +0300 Amichai Rotman wrote: > I know it's kinda off-topic, but I am really at a loss... > > I m trying to free spcace on my Google free storage, so i am sifting > through very old messages to delete in my Gmail box (as far as 2005 > and beyond!) > > I have this one label containing 12,000(!) messages. I'd like to > delete all messages dated before the current year. One easy way, if you can let things run overnight, is to install Claws-Mail, which is pretty darn fast. Point it at your Gmail IMAP, filter out everything before 1/1/2015 (the advanced search thing for selecting just the older messages is "ag" followed by a number of days (the number of days since 12/31/2014. Then highlight them all and run delete. Be sure you set Claws to delete immediately and not leave a ghost copy (I don't know how to explain it any better). Don't try this with Thunderbird. 12K messages with Thunderbird could take several days just to load up. SteveT From amos.shapira at gmail.com Mon Aug 24 04:48:10 2015 From: amos.shapira at gmail.com (Amos Shapira) Date: Mon, 24 Aug 2015 11:48:10 +1000 Subject: [OT] Deleting Thousends of Messages in Gmail In-Reply-To: <20150823212830.76c487a1@mydesq2.domain.cxm> References: <20150823212830.76c487a1@mydesq2.domain.cxm> Message-ID: What would be the advantage of Claws e-mail over the previously provided GMail web interface search? I got the impression the web interface can achieve this on the server side, which will save network bandwidth and time. On 24 August 2015 at 11:28, Steve Litt wrote: > On Sun, 23 Aug 2015 23:07:02 +0300 > Amichai Rotman wrote: > > > I know it's kinda off-topic, but I am really at a loss... > > > > I m trying to free spcace on my Google free storage, so i am sifting > > through very old messages to delete in my Gmail box (as far as 2005 > > and beyond!) > > > > I have this one label containing 12,000(!) messages. I'd like to > > delete all messages dated before the current year. > > One easy way, if you can let things run overnight, is to install > Claws-Mail, which is pretty darn fast. Point it at your Gmail IMAP, > filter out everything before 1/1/2015 (the advanced search thing for > selecting just the older messages is "ag" followed by a number of days > (the number of days since 12/31/2014. Then highlight them all and run > delete. Be sure you set Claws to delete immediately and not leave a > ghost copy (I don't know how to explain it any better). > > Don't try this with Thunderbird. 12K messages with Thunderbird could > take several days just to load up. > > SteveT > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From amichai at iglu.org.il Mon Aug 24 15:11:07 2015 From: amichai at iglu.org.il (Amichai Rotman) Date: Mon, 24 Aug 2015 15:11:07 +0300 Subject: [OT] Deleting Thousends of Messages in Gmail In-Reply-To: References: <20150823212830.76c487a1@mydesq2.domain.cxm> Message-ID: Thank you all, Roman's solution did the trick! I was able to reduce my storage from 91% to 87%!. Amichai. 2015-08-24 4:48 GMT+03:00 Amos Shapira : > What would be the advantage of Claws e-mail over the previously provided > GMail web interface search? > I got the impression the web interface can achieve this on the server > side, which will save network bandwidth and time. > > On 24 August 2015 at 11:28, Steve Litt wrote: > >> On Sun, 23 Aug 2015 23:07:02 +0300 >> Amichai Rotman wrote: >> >> > I know it's kinda off-topic, but I am really at a loss... >> > >> > I m trying to free spcace on my Google free storage, so i am sifting >> > through very old messages to delete in my Gmail box (as far as 2005 >> > and beyond!) >> > >> > I have this one label containing 12,000(!) messages. I'd like to >> > delete all messages dated before the current year. >> >> One easy way, if you can let things run overnight, is to install >> Claws-Mail, which is pretty darn fast. Point it at your Gmail IMAP, >> filter out everything before 1/1/2015 (the advanced search thing for >> selecting just the older messages is "ag" followed by a number of days >> (the number of days since 12/31/2014. Then highlight them all and run >> delete. Be sure you set Claws to delete immediately and not leave a >> ghost copy (I don't know how to explain it any better). >> >> Don't try this with Thunderbird. 12K messages with Thunderbird could >> take several days just to load up. >> >> SteveT >> >> _______________________________________________ >> Linux-il mailing list >> Linux-il at cs.huji.ac.il >> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >> > > > > -- > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From w1 at zak.co.il Tue Aug 25 11:10:18 2015 From: w1 at zak.co.il (Omer Zak) Date: Tue, 25 Aug 2015 11:10:18 +0300 Subject: A gentle introduction to OpenGL, anyone? Message-ID: <1440490218.19045.51.camel@zak.co.il> Does anyone know a book, E-book or Website which contains a gentle introduction to OpenGL - a series of examples with explanations, which go gradually from the easy to sophisticated, eventually covering a large part of the OpenGL API? Preferably using the Qt5.x platform. --- Omer -- PHP - the language of the Vogons. My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html From w1 at zak.co.il Tue Aug 25 12:48:44 2015 From: w1 at zak.co.il (Omer Zak) Date: Tue, 25 Aug 2015 12:48:44 +0300 Subject: A gentle introduction to OpenGL, anyone? In-Reply-To: <1440490218.19045.51.camel@zak.co.il> References: <1440490218.19045.51.camel@zak.co.il> Message-ID: <1440496124.19045.57.camel@zak.co.il> So far I got the following suggestions: * http://nehe.gamedev.net/ * http://www.amazon.com/exec/obidos/ASIN/0321552628/khongrou-20 * http://www.opengl-redbook.com/ (Thanks, Boris Shtrasman and Dov Grobgeld.) However, it seems that the materials describe deprecated OpenGL versions. Specifically, I am looking for materials which also teach about "shader files", which I see in Qt5.5 projects demonstrating OpenGL. On Tue, 2015-08-25 at 11:10 +0300, Omer Zak wrote: > Does anyone know a book, E-book or Website which contains a gentle > introduction to OpenGL - a series of examples with explanations, which > go gradually from the easy to sophisticated, eventually covering a large > part of the OpenGL API? -- The volume of a pizza of thickness a and radius z can be described by the following formula: pi zz a My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html From elazarl at gmail.com Sat Aug 29 22:13:12 2015 From: elazarl at gmail.com (Elazar Leibovich) Date: Sat, 29 Aug 2015 22:13:12 +0300 Subject: HW breakpoint on physical address w/ VM Message-ID: Does anyone have a creative idea how to set a hardware breakpoint on physical address with a VM. In x86-64 architecture. The best idea I've had, is patching KVM, and let him pretend some non-canonical vaddress is actually a physical address (e.g., hbr *0xf00d0A-BCDE-FGHI is equiv to a breakpoint on physical address ABCDEFGHI), but I'll be happy to hear a better approach. (background, someone is messing with my page table, and I can't find out who's messing with my page table, while counting on the page table to report on page table modifications. The guest OS is not exactly linux). -------------- next part -------------- An HTML attachment was scrubbed... URL: From elazarl at gmail.com Sat Aug 29 22:44:17 2015 From: elazarl at gmail.com (Elazar Leibovich) Date: Sat, 29 Aug 2015 22:44:17 +0300 Subject: HW breakpoint on physical address w/ VM In-Reply-To: References: Message-ID: Oh, and the idea of the KVM patch is, for each physical HW bp, add a relevant entry in the spt, and set the hardware breakpoint there. This is assuming KVM HW bp works like I think they do. On Sat, Aug 29, 2015 at 10:13 PM, Elazar Leibovich wrote: > Does anyone have a creative idea how to set a hardware breakpoint on > physical address with a VM. In x86-64 architecture. > > The best idea I've had, is patching KVM, and let him pretend some > non-canonical vaddress is actually a physical address (e.g., hbr > *0xf00d0A-BCDE-FGHI is equiv to a breakpoint on physical address > ABCDEFGHI), but I'll be happy to hear a better approach. > > (background, someone is messing with my page table, and I can't find out > who's messing with my page table, while counting on the page table to > report on page table modifications. The guest OS is not exactly linux). > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mulix at mulix.org Sun Aug 30 07:58:44 2015 From: mulix at mulix.org (Muli Ben-Yehuda) Date: Sun, 30 Aug 2015 07:58:44 +0300 Subject: HW breakpoint on physical address w/ VM In-Reply-To: References: Message-ID: <20150830045844.GA9670@needle> On Sat, Aug 29, 2015 at 10:44:17PM +0300, Elazar Leibovich wrote: > Oh, and the idea of the KVM patch is, for each physical HW bp, add a > relevant entry in the spt, and set the hardware breakpoint > there. This is assuming KVM HW bp works like I think they do. I'm not sure I follow what you are trying to do. But assuming you are working on a guest OS where some code running in guest context is modifying the page tables, assuming you always see the same PTE or the same range of PTEs being modified, I would just set the PTE mapping that PTE page to RO in KVM and wait for the inevitable exit. The stack trace should then point to the culprit. This crude but simple technique has served me well while writing nom (my operating system). Several time when it hadn't, it turned out that my network adapter was DMA'ing directly into memory it wasn't supposed to. Cheers, Muli From erez0001 at gmail.com Sun Aug 30 14:05:06 2015 From: erez0001 at gmail.com (Erez D) Date: Sun, 30 Aug 2015 14:05:06 +0300 Subject: ot: multiple wifi AP for covarage Message-ID: Hello I'm facing with the following problem: one wifi router does not cover the area i need if I put multiple access points. the laptops / smartphones does not switch access points automatically, and i find myself with terrible data rate because the laptop is still connected to a remote AP, while a closer is a meter from me i tried to put all APs with same ESSID, but that no help using a repeater if i understand correctly will not help either (and i get worse throughput) any idea ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at davidsconsultants.com Sun Aug 30 14:33:46 2015 From: david at davidsconsultants.com (David Suna) Date: Sun, 30 Aug 2015 14:33:46 +0300 Subject: HP All-in-one scanner problem Message-ID: <55E2EA1A.5010700@davidsconsultants.com> An HTML attachment was scrubbed... URL: From esr+linux-il at g.jct.ac.il Sun Aug 30 14:50:50 2015 From: esr+linux-il at g.jct.ac.il (E.S. Rosenberg) Date: Sun, 30 Aug 2015 14:50:50 +0300 Subject: HP All-in-one scanner problem In-Reply-To: <55E2EA1A.5010700@davidsconsultants.com> References: <55E2EA1A.5010700@davidsconsultants.com> Message-ID: You may have to set a scan mode on the device too, friends of mine have a Canon all-in-one which needs to be 'told' that a remote device will be in charge of scanning whenever they want to scan.... (Since apparently on windows you can just press the scan button on the scanner and it will open the scanning program on the computer....) HTH, Eliyahu - ????? 2015-08-30 14:33 GMT+03:00 David Suna : > I recently purchased a new HP All-in-one machine model 3835. I checked on > http://hplipopensource.com/ and the model is listed as having full Linux > support. I am running Debian 8.1. I am able to print. However, when I run > scanimage -L it does not find the scanner at all. I tried adding the IP > address of the device to the net.conf file in /etc/sane.d and when I run > scanimage -L with debug messages I get the following: > SANE_DEBUG_NET=255 scanimage -L > [sanei_debug] Setting debug level of net to 255. > [net] sane_init: authorize != null, version_code != null > [net] sane_init: SANE net backend version 1.0.14 (AF-indep+IPv6) from > sane-backends 1.0.24 > [net] sane_init: Client has little endian byte order > [net] sane_init: searching for config file > [net] sane_init: trying to add 10.0.0.24 > [net] add_device: adding backend 10.0.0.24 > [net] net_avahi_browse_callback: CACHE_EXHAUSTED > [net] add_device: backend 10.0.0.24 added > [net] sane_init: done reading config > [net] sane_init: evaluating environment variable SANE_NET_HOSTS > [net] sane_init: evaluating environment variable SANE_NET_TIMEOUT > [net] sane_init: done > [net] sane_get_devices: local_only = 0 > [net] connect_dev: trying to connect to 10.0.0.24 > [net] connect_dev: [0] failed to connect (Connection refused) > [net] connect_dev: couldn't connect to host (see messages above) > [net] sane_get_devices: ignoring failure to connect to 10.0.0.24 > [net] sane_get_devices: finished (0 devices) > [net] net_avahi_browse_callback: ALL_FOR_NOW > > I tried to search for solutions but haven't found anything. Anyone have any > solutions or suggestions of where to look? > > Thanks, > > -- > David Suna > david at davidsconsultants.com > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > From david at davidsconsultants.com Sun Aug 30 14:59:29 2015 From: david at davidsconsultants.com (David Suna) Date: Sun, 30 Aug 2015 14:59:29 +0300 Subject: HP All-in-one scanner problem In-Reply-To: References: <55E2EA1A.5010700@davidsconsultants.com> Message-ID: <55E2F021.9070006@davidsconsultants.com> An HTML attachment was scrubbed... URL: From efraim at flashner.co.il Sun Aug 30 15:35:22 2015 From: efraim at flashner.co.il (Efraim Flashner) Date: Sun, 30 Aug 2015 15:35:22 +0300 Subject: HP All-in-one scanner problem In-Reply-To: <55E2EA1A.5010700@davidsconsultants.com> References: <55E2EA1A.5010700@davidsconsultants.com> Message-ID: <20150830153522.6407a130@debian-netbook> On my system using debian I installed hplip and ran hp-setup from the command line and that's how I dealt with my hp all-in-one. On Sun, Aug 30, 2015 at 02:33:46PM +0300, David Suna wrote: > > > > > > text="#000000" bgcolor="#FFFFFF"> > I recently purchased a new HP All-in-one machine model 3835. I > checked on href="http://hplipopensource.com/">http://hplipopensource.com/ and the > model is listed as having full Linux support. I am running Debian 8.1. I am > able to print. However, when I run scanimage -L it does not find the > scanner at all. I tried adding the IP address of the device to the net.conf > file in /etc/sane.d and when I run scanimage -L with debug messages I get > the following:
SANE_DEBUG_NET=255 scanimage -L
> [sanei_debug] Setting debug level of net to 255.
> [net] sane_init: authorize != null, version_code != null
> [net] sane_init: SANE net backend version 1.0.14 (AF-indep+IPv6) > from sane-backends 1.0.24
> [net] sane_init: Client has little endian byte order
> [net] sane_init: searching for config file
> [net] sane_init: trying to add 10.0.0.24
> [net] add_device: adding backend 10.0.0.24
> [net] net_avahi_browse_callback: CACHE_EXHAUSTED
> [net] add_device: backend 10.0.0.24 added
> [net] sane_init: done reading config
> [net] sane_init: evaluating environment variable SANE_NET_HOSTS
> [net] sane_init: evaluating environment variable SANE_NET_TIMEOUT
> [net] sane_init: done
> [net] sane_get_devices: local_only = 0
> [net] connect_dev: trying to connect to 10.0.0.24
> [net] connect_dev: [0] failed to connect (Connection refused)
> [net] connect_dev: couldn't connect to host (see messages above)
> [net] sane_get_devices: ignoring failure to connect to 10.0.0.24
> [net] sane_get_devices: finished (0 devices)
> [net] net_avahi_browse_callback: ALL_FOR_NOW
>
> I tried to search for solutions but haven't found anything. Anyone > have any solutions or suggestions of where to look?
>
> Thanks,
> 
-- 
> David Suna
>  href="mailto:david at davidsconsultants.com">david at davidsconsultants.com
> > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il -- Efraim Flashner ????? ????? GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From shlomo.solomon at gmail.com Sun Aug 30 18:02:17 2015 From: shlomo.solomon at gmail.com (Shlomo Solomon) Date: Sun, 30 Aug 2015 18:02:17 +0300 Subject: HP All-in-one scanner problem In-Reply-To: <20150830153522.6407a130@debian-netbook> References: <55E2EA1A.5010700@davidsconsultants.com> <20150830153522.6407a130@debian-netbook> Message-ID: <20150830180217.1ecf09e9@shlomo1.solomon> I have an HP 8610, but I assume the drivers and interface are similar. In fact, I didn't even bother installing anything. If you go to http://"your-printer-IP" you get a very nice web interface to setup all aspects o the printer/scanner/fax. In addition, if you chose SCAN from the menu, under Scan to Computer you'll find something called WEBSCAN which allows you to control the scanner from your browser. Hope this helps On Sun, 30 Aug 2015 15:35:22 +0300 Efraim Flashner wrote: > On my system using debian I installed hplip and ran hp-setup from the > command line and that's how I dealt with my hp all-in-one. > > On Sun, Aug 30, 2015 at 02:33:46PM +0300, David Suna wrote: > > > > > > > > > > > text="#000000" bgcolor="#FFFFFF"> > > I recently purchased a new HP All-in-one machine model 3835. I > > checked on > href="http://hplipopensource.com/">http://hplipopensource.com/ > > and the model is listed as having full Linux support. I am running > > Debian 8.1. I am able to print. However, when I run scanimage -L it > > does not find the scanner at all. I tried adding the IP address of > > the device to the net.conf file in /etc/sane.d and when I run > > scanimage -L with debug messages I get the following:
> > SANE_DEBUG_NET=255 scanimage -L
[sanei_debug] Setting debug > > level of net to 255.
[net] sane_init: authorize != null, > > version_code != null
[net] sane_init: SANE net backend version > > 1.0.14 (AF-indep+IPv6) from sane-backends 1.0.24
> > [net] sane_init: Client has little endian byte order
> > [net] sane_init: searching for config file
> > [net] sane_init: trying to add 10.0.0.24
> > [net] add_device: adding backend 10.0.0.24
> > [net] net_avahi_browse_callback: CACHE_EXHAUSTED
> > [net] add_device: backend 10.0.0.24 added
> > [net] sane_init: done reading config
> > [net] sane_init: evaluating environment variable > > SANE_NET_HOSTS
[net] sane_init: evaluating environment variable > > SANE_NET_TIMEOUT
[net] sane_init: done
> > [net] sane_get_devices: local_only = 0
> > [net] connect_dev: trying to connect to 10.0.0.24
> > [net] connect_dev: [0] failed to connect (Connection > > refused)
[net] connect_dev: couldn't connect to host (see > > messages above)
[net] sane_get_devices: ignoring failure to > > connect to 10.0.0.24
[net] sane_get_devices: finished (0 > > devices)
[net] net_avahi_browse_callback: ALL_FOR_NOW
> >
> > I tried to search for solutions but haven't found anything. > > Anyone have any solutions or suggestions of where to look?
> >
> > Thanks,
> > 
-- 
> > David Suna
> >  > href="mailto:david at davidsconsultants.com">david at davidsconsultants.com
> > > > > > > _______________________________________________ > > Linux-il mailing list > > Linux-il at cs.huji.ac.il > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -- Shlomo Solomon http://the-solomons.net Sent by Claws Mail 3.11.1 - KDE 4.12.15 - LINUX Mageia 4 From david at davidsconsultants.com Sun Aug 30 18:07:02 2015 From: david at davidsconsultants.com (David Suna) Date: Sun, 30 Aug 2015 18:07:02 +0300 Subject: HP All-in-one scanner problem In-Reply-To: <20150830180217.1ecf09e9@shlomo1.solomon> References: <55E2EA1A.5010700@davidsconsultants.com> <20150830153522.6407a130@debian-netbook> <20150830180217.1ecf09e9@shlomo1.solomon> Message-ID: <55E31C16.20004@davidsconsultants.com> An HTML attachment was scrubbed... URL: From shlomo.solomon at gmail.com Sun Aug 30 18:18:46 2015 From: shlomo.solomon at gmail.com (Shlomo Solomon) Date: Sun, 30 Aug 2015 18:18:46 +0300 Subject: HP All-in-one scanner problem In-Reply-To: <55E31C16.20004@davidsconsultants.com> References: <55E2EA1A.5010700@davidsconsultants.com> <20150830153522.6407a130@debian-netbook> <20150830180217.1ecf09e9@shlomo1.solomon> <55E31C16.20004@davidsconsultants.com> Message-ID: <20150830181846.53cf05ad@shlomo1.solomon> Just curious - can you fax from applications? Personally I never tried it - I do it the "old fashioned" way - from the scanner glass or ADF. On Sun, 30 Aug 2015 18:07:02 +0300 David Suna wrote: > Thanks that works. Still not sure why I cannot find the scanner using > sane. That would allow scanning directly into applications (like > GIMP) rather than scanning in the browser and saving the file and > opening it in the program. > > On 08/30/2015 06:02 PM, Shlomo Solomon wrote: > I have an HP 8610, but I assume the drivers and interface are similar. > > In fact, I didn't even bother installing anything. If you go to > http://"your-printer-IP" you get a very nice web interface to setup > all aspects o the printer/scanner/fax. > > In addition, if you chose SCAN from the menu, under Scan to Computer > you'll find something called WEBSCAN which allows you to control the > scanner from your browser. > > Hope this helps > > > > On Sun, 30 Aug 2015 15:35:22 +0300 > Efraim Flashner wrote: > > On my system using debian I installed hplip and ran hp-setup from the > command line and that's how I dealt with my hp all-in-one. > > On Sun, Aug 30, 2015 at 02:33:46PM +0300, David Suna wrote: > > > > > text="#000000" bgcolor="#FFFFFF"> > I recently purchased a new HP All-in-one machine model 3835. I > checked on href="http://hplipopensource.com/">http://hplipopensource.com/ > and the model is listed as having full Linux support. I am running > Debian 8.1. I am able to print. However, when I run scanimage -L it > does not find the scanner at all. I tried adding the IP address of > the device to the net.conf file in /etc/sane.d and when I run > scanimage -L with debug messages I get the following:
> SANE_DEBUG_NET=255 scanimage -L
[sanei_debug] Setting debug > level of net to 255.
[net] sane_init: authorize != null, > version_code != null
[net] sane_init: SANE net backend version > 1.0.14 (AF-indep+IPv6) from sane-backends 1.0.24
> [net] sane_init: Client has little endian byte order
> [net] sane_init: searching for config file
> [net] sane_init: trying to add 10.0.0.24
> [net] add_device: adding backend 10.0.0.24
> [net] net_avahi_browse_callback: CACHE_EXHAUSTED
> [net] add_device: backend 10.0.0.24 added
> [net] sane_init: done reading config
> [net] sane_init: evaluating environment variable > SANE_NET_HOSTS
[net] sane_init: evaluating environment variable > SANE_NET_TIMEOUT
[net] sane_init: done
> [net] sane_get_devices: local_only = 0
> [net] connect_dev: trying to connect to 10.0.0.24
> [net] connect_dev: [0] failed to connect (Connection > refused)
[net] connect_dev: couldn't connect to host (see > messages above)
[net] sane_get_devices: ignoring failure to > connect to 10.0.0.24
[net] sane_get_devices: finished (0 > devices)
[net] net_avahi_browse_callback: ALL_FOR_NOW
>
> I tried to search for solutions but haven't found anything. > Anyone have any solutions or suggestions of where to look?
>
> Thanks,
> 
-- 
> David Suna
>  href="mailto:david at davidsconsultants.com">david at davidsconsultants.com
> > > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > > > > > -- Shlomo Solomon http://the-solomons.net Sent by Claws Mail 3.11.1 - KDE 4.12.15 - LINUX Mageia 4 From lior at okman.name Sun Aug 30 21:29:13 2015 From: lior at okman.name (Lior Okman) Date: Sun, 30 Aug 2015 21:29:13 +0300 Subject: ot: multiple wifi AP for covarage In-Reply-To: References: Message-ID: On Sun, Aug 30, 2015 at 2:05 PM, Erez D wrote: > Hello > > I'm facing with the following problem: > one wifi router does not cover the area i need > if I put multiple access points. the laptops / smartphones does not switch > access points automatically, and i find myself with terrible data rate > because the laptop is still connected to a remote AP, while a closer is a > meter from me > > i tried to put all APs with same ESSID, but that no help > I think that it's not enough that all the APs have the same ESSID. You also have to verify that the security settings are exactly the same on all of the access points, and that each access point is using a different channel. > using a repeater if i understand correctly will not help either (and i get > worse throughput) > > > any idea ? > > _______________________________________________ > Linux-il mailing list > Linux-il at cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From baruch at shpirer.com Sun Aug 30 21:51:58 2015 From: baruch at shpirer.com (Baruch Shpirer) Date: Sun, 30 Aug 2015 14:51:58 -0400 Subject: ot: multiple wifi AP for covarage In-Reply-To: References: Message-ID: <030701d0e354$f2fd29e0$d8f77da0$@shpirer.com> If not using roaming under WDS or repeater/expander you can use AP with same SSID and configure roaming aggressiveness to medium/high See example: http://www.tp-link.com/en/faq-592.html From: Linux-il [mailto:linux-il-bounces at cs.huji.ac.il] On Behalf Of Lior Okman Sent: August-30-15 14:29 To: Erez D Cc: linux-il Subject: Re: ot: multiple wifi AP for covarage On Sun, Aug 30, 2015 at 2:05 PM, Erez D wrote: Hello I'm facing with the following problem: one wifi router does not cover the area i need if I put multiple access points. the laptops / smartphones does not switch access points automatically, and i find myself with terrible data rate because the laptop is still connected to a remote AP, while a closer is a meter from me i tried to put all APs with same ESSID, but that no help I think that it's not enough that all the APs have the same ESSID. You also have to verify that the security settings are exactly the same on all of the access points, and that each access point is using a different channel. using a repeater if i understand correctly will not help either (and i get worse throughput) any idea ? _______________________________________________ Linux-il mailing list Linux-il at cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at davidsconsultants.com Sun Aug 30 22:21:46 2015 From: david at davidsconsultants.com (David Suna) Date: Sun, 30 Aug 2015 22:21:46 +0300 Subject: HP All-in-one scanner problem In-Reply-To: <20150830181846.53cf05ad@shlomo1.solomon> References: <55E2EA1A.5010700@davidsconsultants.com> <20150830153522.6407a130@debian-netbook> <20150830180217.1ecf09e9@shlomo1.solomon> <55E31C16.20004@davidsconsultants.com> <20150830181846.53cf05ad@shlomo1.solomon> Message-ID: <55E357CA.2070106@davidsconsultants.com> An HTML attachment was scrubbed... URL: From alon at skylinesoft.com Mon Aug 31 18:44:49 2015 From: alon at skylinesoft.com (Alon Barzilai) Date: Mon, 31 Aug 2015 18:44:49 +0300 Subject: ot: multiple wifi AP for covarage In-Reply-To: References: Message-ID: <55E47671.7080106@skylinesoft.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: skyline.png Type: image/png Size: 4826 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pixel.png Type: image/png Size: 935 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: linkedin.png Type: image/png Size: 1195 bytes Desc: not available URL: