Glusterfs blues

Issues related to applications and software problems

Glusterfs blues

Postby hpux » 2012/02/07 05:45:58

Installed glusterfs from epel had it working fairly well.

Installed some other rpms to make multimedia work better.

Now glusterfs does not want to work.

Feb 6 22:29:25 solaria abrtd: Package 'glusterfs' isn't signed with proper key
Feb 6 22:29:25 solaria abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-02-06-22:29:24-4090 (res:2), deleting


Crashes every time I start the glusterd daemon.

Should I have installed from epel or is this now part of the core functionality of the OS?
hpux
 
Posts: 25
Joined: 2012/02/07 05:42:30

Glusterfs blues

Postby pschaff » 2012/02/07 18:59:13

EPEL has a policy of not replacing distro packages, and they are the only source for glusterfs ins of the Repositories page. What repo did you use for multimedia packages? What does "rpm -V glusterfs" show?

Please provide more information about your system by running "./getinfo.sh package" and showing us the output file.
pschaff
Retired Moderator
 
Posts: 18277
Joined: 2006/12/13 20:15:34
Location: Tidewater, Virginia, North America

Re: Glusterfs blues

Postby hpux » 2012/02/08 02:01:26

Information for general problems.
[code]
== BEGIN uname -rmi ==
2.6.32-220.4.1.el6.x86_64 x86_64 x86_64
== END uname -rmi ==

== BEGIN rpm -qa \*-release\* ==
epel-release-6-5.noarch
centos-release-6-2.el6.centos.7.x86_64
adobe-release-x86_64-1.0-1.noarch
rpmforge-release-0.5.2-2.el6.rf.x86_64
== END rpm -qa \*-release\* ==

== BEGIN cat /etc/redhat-release ==
CentOS release 6.2 (Final)
== END cat /etc/redhat-release ==

== BEGIN getenforce ==
Enforcing
== END getenforce ==

== BEGIN free -m ==
total used free shared buffers cached

used atrpms to do the multi media stuff.

Backed some of it out but not completely.

I've disabled the repo and yum removed and yum installed glusterfs

Still no joy.

ran yum clean all after yum remove.

No joy after re-install.

[root@solaria bin]# rpm -qa | grep gluster
glusterfs-vim-3.2.5-6.el6.x86_64
glusterfs-3.2.5-6.el6.x86_64
glusterfs-server-3.2.5-6.el6.x86_64
glusterfs-rdma-3.2.5-6.el6.x86_64
glusterfs-fuse-3.2.5-6.el6.x86_64
glusterfs-geo-replication-3.2.5-6.el6.x86_64


[root@solaria bin]# uname -a
Linux solaria 2.6.32-220.4.1.el6.x86_64 #1 SMP Tue Jan 24 02:13:44 GMT 2012 x86_64 x86_64 x86_64 GNU/Linux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

Feb 7 20:49:13 solaria GlusterFS[4079]: [2012-02-07 20:49:13.141263] C [rdma.c:3934:rdma_init] 0-rpc-transport/rdma: Failed to get IB devices
Feb 7 20:49:16 solaria abrt[4124]: saved core dump of pid 4079 (/usr/sbin/glusterfsd) to /var/spool/abrt/ccpp-2012-02-07-20:49:16-4079.new/coredump (44335104 bytes)
Feb 7 20:49:16 solaria abrtd: Directory 'ccpp-2012-02-07-20:49:16-4079' creation detected
Feb 7 20:49:17 solaria abrtd: Package 'glusterfs' isn't signed with proper key
Feb 7 20:49:17 solaria abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-02-07-20:49:16-4079 (res:2), deleting


[root@solaria ccpp-2012-02-07-05:49:21-4586]# ll
total 32832
-rw-r-----. 1 abrt invest 4 Feb 7 05:49 analyzer
-rw-r-----. 1 abrt invest 6 Feb 7 05:49 architecture
-rw-r-----. 1 abrt invest 30 Feb 7 05:49 cmdline
-rw-r-----. 1 abrt invest 7 Feb 7 05:49 component
-rw-r-----. 1 abrt invest 297582592 Feb 7 05:49 coredump
-rw-r-----. 1 abrt invest 1 Feb 7 05:49 count
-rw-r-----. 1 abrt invest 14844 Feb 7 05:49 dso_list
-rw-r-----. 1 abrt invest 2155 Feb 7 05:49 environ
-rw-r-----. 1 abrt invest 30 Feb 7 05:49 executable
-rw-r-----. 1 abrt invest 7 Feb 7 05:49 hostname
-rw-r-----. 1 abrt invest 25 Feb 7 05:49 kernel
-rw-r-----. 1 abrt invest 63386 Feb 7 05:49 maps
-rw-r-----. 1 abrt invest 26 Feb 7 05:49 os_release
-rw-r-----. 1 abrt invest 27 Feb 7 05:49 package
-rw-r-----. 1 abrt invest 71 Feb 7 05:49 reason
-rw-r-----. 1 abrt invest 22 Feb 7 05:49 reported_to
-rw-r-----. 1 abrt invest 1309744 Feb 7 05:49 sosreport.tar.xz
-rw-r-----. 1 abrt invest 10 Feb 7 05:49 time
-rw-r-----. 1 abrt invest 3 Feb 7 05:49 uid
-rw-r-----. 1 abrt invest 7 Feb 7 05:49 username
-rw-r-----. 1 abrt invest 40 Feb 7 05:49 uuid
-rw-r-----. 1 abrt invest 178 Feb 7 05:49 var_log_messages
[root@solaria ccpp-2012-02-07-05:49:21-4586]#
[root@solaria ccpp-2012-02-07-05:49:21-4586]# pwd
/var/spool/abrt/ccpp-2012-02-07-05:49:21-4586

glusterfs -V
glusterfs 3.2.5 built on Jan 10 2012 18:19:36
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc.
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
hpux
 
Posts: 25
Joined: 2012/02/07 05:42:30

Re: Glusterfs blues

Postby hpux » 2012/02/08 03:14:31

crash dump
hpux
 
Posts: 25
Joined: 2012/02/07 05:42:30

Re: Glusterfs blues

Postby hpux » 2012/02/08 17:44:56

This is a very vanilla 64 bit Centos system.

I have two 6.2 64 bit VM's on the bugger and glusterfs works on them.

I can't believe configuring the system to be able to watch videos (or rather listen) while I work could cause this.

I'm about ready to trash the system and start over. Something else funky could have happened here.

Anyone have a clue on what I can do besides re-install? I can take care of backing up the data and imaging the system in an hour using kickstart but its quite irritating.
hpux
 
Posts: 25
Joined: 2012/02/07 05:42:30

Re: Glusterfs blues

Postby pschaff » 2012/02/08 18:23:54

Reinstalling is usually overkill, but is at least guaranteed to get you back to a deterministic state. You did not show but the beginning of the requested "./getinfo.sh package" output so it is not clear how your repos are configured.

To see what repos you have installed packages from:
Code: Select all
rpm -qa --qf "%{VENDOR} \n" | sort | uniq

The result should be a list of VENDOR tags. To see the packages from a particular Vendor, for example EPEL:
Code: Select all
rpm -qa --qf "%{NAME} %{VENDOR} \n" | grep "Fedora Project" | cut -d ' ' -f 1 | sort

The result should be a list of EPEL packages.

To show RPMforge packages:
Code: Select all
rpm -qa --qf "%{NAME} %{VENDOR} \n" | grep "Dag Apt" | cut -d ' ' -f 1 | sort

To show ATrpms packages:
Code: Select all
rpm -qa --qf "%{NAME} %{VENDOR} \n" | grep ATrpms | cut -d ' ' -f 1 | sort


Removing the ATrpms packages, if you suspect that is the problem, may be worth trying. If core packages have been replaced you may instead need to downgrade to the CentOS versions.
pschaff
Retired Moderator
 
Posts: 18277
Joined: 2006/12/13 20:15:34
Location: Tidewater, Virginia, North America

Re: Glusterfs blues

Postby hpux » 2012/02/09 02:24:07

########################
Results are in the file:
* /tmp/basedata.oNXlxf *
########################

To provide this information in support of a question on the CentOS fora,
mailing list, or IRC channel please post its contents to

http://pastebin.centos.org/

or another similar public site of your choice.
Select a retention time longer than the default of 'a day'
and provide a link to the information in your post.

Alternatively, post in-line, trimming to remove any unnecessary information
for your topic. For forum posts, please leave in the
Code: Select all
 ...
tags to preserve formatting.

WARNING - Check the contents of the /tmp/basedata.oNXlxf file to ensure it does
not contain any sensitive information such as public IP addresses or hostnames.
If necessary obfuscate such information before posting. For example:
146.xxx.xxx.41 or myhost.mydomain.net
[root@solaria bin]# cat /tmp/basedata.oNXlxf
Information for package problems.
Code: Select all
== BEGIN uname -rmi ==
2.6.32-220.4.1.el6.x86_64 x86_64 x86_64
== END   uname -rmi ==

== BEGIN rpm -qa \*-release\* ==
epel-release-6-5.noarch
centos-release-6-2.el6.centos.7.x86_64
adobe-release-x86_64-1.0-1.noarch
rpmforge-release-0.5.2-2.el6.rf.x86_64
== END   rpm -qa \*-release\* ==

== BEGIN cat /etc/redhat-release ==
CentOS release 6.2 (Final)
== END   cat /etc/redhat-release ==

== BEGIN getenforce ==
Enforcing
== END   getenforce ==

== BEGIN free -m ==
             total       used       free     shared    buffers     cached
Mem:          5759       1703       4055          0         36        388
-/+ buffers/cache:       1278       4480
Swap:        16383          0      16383
== END   free -m ==

== BEGIN rpm -qa yum\* rpm-\* python | sort ==
python-2.6.6-29.el6.x86_64
rpm-build-4.8.0-19.el6.x86_64
rpm-libs-4.8.0-19.el6.x86_64
rpm-python-4.8.0-19.el6.x86_64
yum-3.2.29-22.el6.centos.noarch
yum-metadata-parser-1.1.2-16.el6.x86_64
yum-plugin-fastestmirror-1.1.30-10.el6.noarch
yum-plugin-security-1.1.30-10.el6.noarch
yum-utils-1.1.30-10.el6.noarch
== END   rpm -qa yum\* rpm-\* python | sort ==

== BEGIN ls /etc/yum.repos.d ==
adobe-linux-x86_64.repo
atrpms-bleeding.repo
atrpms.repo
atrpms-testing.repo
CentOS-Base.repo
CentOS-Debuginfo.repo
CentOS-Media.repo
epel.repo
epel-testing.repo
google-chrome.repo
google.repo
mirrors-rpmforge
mirrors-rpmforge-extras
mirrors-rpmforge-testing
rpmforge.repo
== END   ls /etc/yum.repos.d ==

== BEGIN cat /etc/yum.conf ==
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=16&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release

#  This is the default, if you make this bigger yum won't see if the metadata
# is newer on the remote and so you'll "gain" the bandwidth of not having to
# download the new metadata and "pay" for it by yum not having correct
# information.
#  It is esp. important, to have correct metadata, for distributions like
# Fedora which don't keep old packages around. If you don't like this checking
# interupting your command line usage, it's much better to have something
# manually check the metadata once an hour (yum-updatesd will do this).
# metadata_expire=90m

# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum.repos.d
== END   cat /etc/yum.conf ==

== BEGIN yum repolist all ==
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: mirror.nexcess.net
 * epel: mirrors.ptd.net
 * extras: mirror.steadfast.net
 * rpmforge: fr2.rpmfind.net
 * updates: pubmirrors.reflected.net
repo id                   repo name                               status
adobe-linux-x86_64        Adobe Systems Incorporated              enabled:     2
atrpms                    Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-bleeding           Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-bleeding-debuginfo Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-bleeding-source    Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-debuginfo          Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-source             Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-testing            Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-testing-debuginfo  Red Hat Enterprise Linux 6 - x86_64 - A disabled
atrpms-testing-source     Red Hat Enterprise Linux 6 - x86_64 - A disabled
base                      CentOS-6 - Base                         enabled: 6,294
c6-media                  CentOS-6 - Media                        disabled
centosplus                CentOS-6 - Plus                         disabled
contrib                   CentOS-6 - Contrib                      disabled
debug                     CentOS-6 - Debuginfo                    disabled
epel                      Extra Packages for Enterprise Linux 6 - enabled: 6,916
epel-debuginfo            Extra Packages for Enterprise Linux 6 - disabled
epel-source               Extra Packages for Enterprise Linux 6 - disabled
epel-testing              Extra Packages for Enterprise Linux 6 - disabled
epel-testing-debuginfo    Extra Packages for Enterprise Linux 6 - disabled
epel-testing-source       Extra Packages for Enterprise Linux 6 - disabled
exodus                    exodus-viewer                           enabled:     3
extras                    CentOS-6 - Extras                       enabled:     4
google-chrome             google-chrome - 64-bit                  enabled:     3
rpmforge                  RHEL 6 - RPMforge.net - dag             enabled: 4,197
rpmforge-extras           RHEL 6 - RPMforge.net - extras          disabled
rpmforge-testing          RHEL 6 - RPMforge.net - testing         disabled
updates                   CentOS-6 - Updates                      enabled:   338
repolist: 17,757
== END   yum repolist all ==

== BEGIN egrep 'include|exclude' /etc/yum.repos.d/*.repo ==
== END   egrep 'include|exclude' /etc/yum.repos.d/*.repo ==

== BEGIN sed -n -e "/^\[/h; /priority *=/{ G; s/\n/ /; s/ity=/ity = /; p }" /etc/yum.repos.d/*.repo | sort -k3n ==
== END   sed -n -e "/^\[/h; /priority *=/{ G; s/\n/ /; s/ity=/ity = /; p }" /etc/yum.repos.d/*.repo | sort -k3n ==

== BEGIN rpm -qa kernel\* | sort ==
kernel-2.6.32-220.2.1.el6.x86_64
kernel-2.6.32-220.4.1.el6.x86_64
kernel-2.6.32-220.el6.x86_64
kernel-devel-2.6.32-220.2.1.el6.x86_64
kernel-devel-2.6.32-220.4.1.el6.x86_64
kernel-devel-2.6.32-220.el6.x86_64
kernel-firmware-2.6.32-220.4.1.el6.noarch
kernel-headers-2.6.32-220.4.1.el6.x86_64
== END   rpm -qa kernel\* | sort ==
[Moderator edit: Restored code tag to preserve formatting.]

[root@solaria bin]# rpm -qa --qf "%{VENDOR} \n" | sort | uniq
Adobe Systems Inc.
ATrpms.net
CentOS
Dag Apt Repository, http://dag.wieers.com/apt/
Fedora Project
Fujitsu Limited
Google Inc.
(none)
Red Hat
Red Hat, Inc.

[root@solaria bin]# rpm -qa --qf "%{NAME} %{VENDOR} \n" | grep "Fedora Project" | cut -d ' ' -f 1 | sort
dirac-libs
directfb
epel-release
jack-audio-connection-kit
libcdaudio
libkate
libmodplug
mod_security
openvpn
openvpn-auth-ldap
portaudio
transmission
transmission-cli
transmission-common
transmission-gtk
[root@solaria bin]#


[root@solaria bin]# clear;rpm -qa --qf "%{NAME} %{VENDOR} \n" | grep "Dag Apt" | cut -d ' ' -f 1 | sort





a52dec
aalib
amrwb
glib
libdvdplay
libmad
libmpeg2
librtmp
libsidplay
lsdvd
orc
pkcs11-helper
rpmforge-release
schroedinger
twolame


[root@solaria bin]#
[root@solaria bin]# rpm -qa --qf "%{NAME} %{VENDOR} \n" | grep ATrpms | cut -d ' ' -f 1 | sort
atrpms-repo
enca
gecko-mediaplayer
gnome-mplayer
gstreamer-plugins-ugly
gstreamer-vaapi
gtk+
libavcodec52
libavutil50
libcaca
libdca0
libdvdcss
libdvdcss2
libdvdnav
libfaac0
libfaad2
liblirc_client0
libmikmod
libmms
libmp3lame0
libnemesi
libopencore-amrnb0
libopencore-amrwb0
libquicktime0
libva1
libva-glx1
libva-x11_1
libvdpau1
libx264_115
libx264_118
libxavs1
libxosd2
libxvidcore4
mjpegtools
mpg123
mplayer
mplayer-fonts
mplayer-skins
nas-libs
netembryo
openal-soft
svgalib
tslib
xmms-libs
hpux
 
Posts: 25
Joined: 2012/02/07 05:42:30

Re: Glusterfs blues

Postby pschaff » 2012/02/09 14:09:37

We really did not need to see the screen output from the script, and the yum diagnostics were meant for your consumption, but do show quite a dangerous mix of packages. Without careful use of yum-plugin-priorities (which is not installed at all) and/or considerably repo tuning with includepkgs/exclude, that mix is of 3rd party repos is pretty much a guaranteed recipe for disaster.

Frankly, if you want to run a cluster filesystem, and presumably a cluster, I'd leave all the multimedia stuff out of the mix and use a separate system for that.

A reinstall would be the cleanest path forward, but to attempt fix things at this point, I'd probably try removing all the 3rd party packages, disabling all 3rd party repos, reinstalling anything necessary that was removed along with the 3rd party packages, enabling only EPEL, and reinstalling glusterfs.
pschaff
Retired Moderator
 
Posts: 18277
Joined: 2006/12/13 20:15:34
Location: Tidewater, Virginia, North America

Re: Glusterfs blues

Postby hpux » 2012/02/09 19:21:57

Well I'll get yum-priorities installed ASAP.

Other advice welcome. I re-imaged the box but am wondering if there is a better source for glusterfs than epel. I thought RH built that into the base.

So far glusterfs is stable
hpux
 
Posts: 25
Joined: 2012/02/07 05:42:30

Re: Glusterfs blues

Postby hpux » 2012/02/09 19:43:32

What do you think about rpmforce?

A monitor free box with a near identical setup (and yum-priorities) running in my basement without a monitor (negating the need to entertain the owner) remained stable.



hpux wrote:
Well I'll get yum-priorities installed ASAP.

Other advice welcome. I re-imaged the box but am wondering if there is a better source for glusterfs than epel. I thought RH built that into the base.

So far glusterfs is stable
hpux
 
Posts: 25
Joined: 2012/02/07 05:42:30

Next

Return to CentOS 6 - Software Support

Who is online

Users browsing this forum: No registered users and 5 guests