Need help installing Kafka components, pretty please

Issues related to applications and software problems
Post Reply
phloks
Posts: 2
Joined: 2018/06/07 12:57:00

Need help installing Kafka components, pretty please

Post by phloks » 2018/06/07 13:08:01

Hi,

I'm running CentOS Linux release 7.5.1804 (Core), and was able to install Kafka from the official site.
However, I need to monitor a syslog file and put the contents of this file in a kafka topic.
There is a kafka connector available that does this for me, but it is not anywhere to be found in the CentOS repo's.

I'm looking to install rsyslog-kafka (that requires librdkafka, which I could install without any problems), but there seems to be a mismatch
in versions and rsyslog-kafka refuses to install, even with --skip-broken.

Code: Select all

yum localinstall rsyslog-kafka-8.24.0-2.el7.x86_64.rpm
Loaded plugins: fastestmirror
Examining rsyslog-kafka-8.24.0-2.el7.x86_64.rpm: rsyslog-kafka-8.24.0-2.el7.x86_64
Marking rsyslog-kafka-8.24.0-2.el7.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package rsyslog-kafka.x86_64 0:8.24.0-2.el7 will be installed
--> Processing Dependency: rsyslog = 8.24.0-2.el7 for package: rsyslog-kafka-8.24.0-2.el7.x86_64
Loading mirror speeds from cached hostfile
--> Finished Dependency Resolution
Error: Package: rsyslog-kafka-8.24.0-2.el7.x86_64 (/rsyslog-kafka-8.24.0-2.el7.x86_64)
           Requires: rsyslog = 8.24.0-2.el7
           Installed: rsyslog-8.24.0-16.el7_5.4.x86_64 (@updates)
               rsyslog = 8.24.0-16.el7_5.4
           Available: rsyslog-8.24.0-16.el7.x86_64 (base)
               rsyslog = 8.24.0-16.el7
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
Is there any way around this ?
Anyone here succeeded in installing this piece of software without installing rsyslog from the rsyslog site, instead of the rsyslog from the CentOS repo ?

I was unable to find an rsyslog-kafka-8.24.0-16.el7.x86_64 to match the installed rsyslog version.....

Thanks,
Hans

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Need help installing Kafka components, pretty please

Post by TrevorH » 2018/06/07 14:34:37

Unfortunately rsyslog-kafka is not a package we provide and the rest of our rsyslog packages are now all 8.24.0-16.el7_5.4 so you need to find out who provided that foreign package and get them to rebuild against the latest. The one you have requires an explicit version of rsyslog so there is no other choice - the package has to be rebuilt to run with the latest rsyslog that we provide.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

phloks
Posts: 2
Joined: 2018/06/07 12:57:00

Re: Need help installing Kafka components, pretty please

Post by phloks » 2018/06/08 08:26:43

Hi Trevor,

Thank you for your answer.
Any particular reason why rsyslog-kafka is not in the CentOS repo ? I would have thought that this is a piece of software that is
widely used. I wouldn't know of any other software that can do the same reliably. There are some (unmaintained) efforts on Github, but I intend to
use it in a production environment eventually, so that is not really an option.
Then again, you can't have everything in the repo's, can you ?
I think that my best and simplest option is to install the latest version of rsyslog from their site, along with the rsyslog-kafka of the same version they provide. Just have to be awake when I do a yum update.... :cry:

Thanks,
Hans

Post Reply