{"id":6938,"date":"2020-04-17T12:33:26","date_gmt":"2020-04-17T12:33:26","guid":{"rendered":"https:\/\/cloudkul.com\/blog\/?p=6938"},"modified":"2020-04-17T13:10:21","modified_gmt":"2020-04-17T13:10:21","slug":"use-apache-kafka-with-php","status":"publish","type":"post","link":"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/","title":{"rendered":"Use Apache Kafka with PHP"},"content":{"rendered":"\n<p>Previously, we got <a rel=\"noreferrer noopener\" aria-label=\"Kafka setup and configuration in our Ubuntu 18.04 machine (opens in a new tab)\" href=\"https:\/\/cloudkul.com\/blog\/apache-kafka-implementation\/\" target=\"_blank\">Kafka setup and configuration in our Ubuntu 18.04 machine<\/a>. Now we will go for Apache Kafka with PHP which is use to produce as well as consume the messages.<\/p>\n\n\n\n<p>By the help of Kafka we can transfer continuously streaming data to the cluster, for example history of website visits, financial transactions, online shopping order, application logs, etc, use machine learning or aggregate data for further analysis. This all takes seconds or minutes, instead of hours and days.<\/p>\n\n\n\n<p>You can read more in the <a href=\"https:\/\/cloudkul.com\/blog\/apache-kafka\/\">Apache Kafka Page<\/a>.<\/p>\n\n\n\n<p>Now we start with Apache Kafka with PHP, before that you might be follow <a href=\"https:\/\/cloudkul.com\/blog\/apache-kafka-implementation\/\">this link<\/a> to install zookeeper and Apache Kafka in Ubuntu 18.04. If you already done with installation then follow the steps.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What we will do?<\/h3>\n\n\n\n<ul><li>Install PHP Client<\/li><li>Create Consumer<\/li><li>Create Producer<\/li><li>Test Environment<\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Install PHP Client :-<\/h4>\n\n\n\n<p>There are several PHP clients for Kafka. I recommend using this <a href=\"https:\/\/github.com\/weiboad\/kafka-php\">Compose<\/a>, because it shows better performance than some other PHP clients.<\/p>\n\n\n\n<p>Now go to same folder in which Kafka is configured and run the command.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">compose required nmred\/kafka-php<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Create Consumer :-<\/h4>\n\n\n\n<p>Create file &#8220;Consumer.php&#8221; with any editor and set broker to &#8220;127.0.0.1:9092&#8221; as Kafka cluster is at local with one broker (node).<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">vi Consumer.php<\/pre>\n\n\n\n<p>Add the following colored content to file.<\/p>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>&lt;?php<br>\nrequire &#8216;vendor\/autoload.php&#8217;;<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>$config = \\Kafka\\ConsumerConfig::getInstance();<br> $config-&gt;setMetadataRefreshIntervalMs(10000);<br> $config-&gt;setMetadataBrokerList(&#8216;127.0.0.1:9092&#8217;);<br> $config-&gt;setGroupId(&#8216;test&#8217;);<br> $config-&gt;setBrokerVersion(&#8216;1.0.0&#8217;);<br> $config-&gt;setTopics([&#8216;test&#8217;]);<br> $consumer = new \\Kafka\\Consumer();<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>$consumer-&gt;start(function($topic, $part, $message) {<br>     var_dump($message);<br> });<\/p>\n<\/div><\/div>\n\n\n\n<p>Save as well as exit from the file with &#8220;esc&#8221; + &#8220;:wq!&#8221;.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Create Producer :-<\/h4>\n\n\n\n<p>A producer works on two modes &#8211; Asynchronous (Single message at a time) and Synchronous (Multiple message at a time).<\/p>\n\n\n\n<p>For Asynchronous message create file Producer.php in any editor.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">vi Producer.php<\/pre>\n\n\n\n<p>Add the following colored content in file.<\/p>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>&lt;?php <br>require &#8216;vendor\/autoload.php&#8217;;<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>$config = \\Kafka\\ProducerConfig::getInstance(); <br>$config-&gt;setMetadataRefreshIntervalMs(10000); <br>$config-&gt;setMetadataBrokerList(&#8216;127.0.0.1:9092&#8217;); <br>$config-&gt;setBrokerVersion(&#8216;1.0.0&#8217;); $config-&gt;setRequiredAck(1); <br>$config-&gt;setIsAsyn(false); $config-&gt;setProduceInterval(500);<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>$producer = new \\Kafka\\Producer(<br>     function() {<br>         return [<br>             [<br>                 &#8216;topic&#8217; =&gt; &#8216;test&#8217;,<br>                 &#8216;value&#8217; =&gt; &#8216;Cloudkul&#8230;.message.&#8217;,<br>                 &#8216;key&#8217; =&gt; &#8216;testkey&#8217;,<br>             ],<br>         ];<br>     }<br> );<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>$producer-&gt;success(function($result) {<br>     var_dump($result); }); <br>$producer-&gt;error(function($errorCode) {<br>     var_dump($errorCode); <br>});<br>$producer-&gt;send(true);<\/p>\n<\/div><\/div>\n\n\n\n<p> Save as well as exit from the file with &#8220;esc&#8221; + &#8220;:wq!&#8221;. <\/p>\n\n\n\n<p> For Synchronous message create file ProducerSyc.php in any editor.  Where the producer produce 200 message at time.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">vi ProducerSyn.php<\/pre>\n\n\n\n<p>Add the following colored content in the file.<\/p>\n\n\n\n<div class=\"wp-block-group has-light-green-cyan-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>&lt;?php <br>require &#8216;vendor\/autoload.php&#8217;; <br>$config = \\Kafka\\ProducerConfig::getInstance(); <br>$config-&gt;setMetadataRefreshIntervalMs(10000); <br>$config-&gt;setMetadataBrokerList(&#8216;127.0.0.1:9092&#8217;); <br>$config-&gt;setBrokerVersion(&#8216;1.0.0&#8217;); <br>$config-&gt;setRequiredAck(1); <br>$config-&gt;setIsAsyn(false); <br>$config-&gt;setProduceInterval(500); <br>$producer = new \\Kafka\\Producer(); <br>for($i = 0; $i &lt; 200; $i++) {<br>     $producer-&gt;send([<br>         [<br>             &#8216;topic&#8217; =&gt; &#8216;test&#8217;,<br>             &#8216;value&#8217; =&gt; &#8216;Cloudkul&#8230; message #&#8217;.$i,<br>             &#8216;key&#8217; =&gt; &#8221;,<br>         ],<br>     ]);<br> }<\/p>\n<\/div><\/div>\n\n\n\n<p> Save as well as exit from the file with &#8220;esc&#8221; + &#8220;:wq!&#8221;. <\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Test Environment :-<\/h4>\n\n\n\n<p>Now we have all the required setup in our Ubuntu 18.04, so first we start our Consumer, run in the terminal.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">php Consumer.php<\/pre>\n\n\n\n<p>Then from another terminal we will try to send message with the help of Producer.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">php Producer.php<\/pre>\n\n\n\n<p>And also check the Synchronous Producer.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">php ProducerSyc.php<\/pre>\n\n\n\n<p>Now we see in the consumer terminal then we find all the messages are found their, exactly what we want.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Previously, we got Kafka setup and configuration in our Ubuntu 18.04 machine. Now we will <a class=\"text-primary\" title=\"read more\" href=\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/\">[&#8230;]<\/a><\/p>\n","protected":false},"author":33,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Use Apache Kafka with PHP - Cloudkul Use Apache Kafka with PHP<\/title>\n<meta name=\"description\" content=\"By the help of Kafka we can transfer continuously streaming data to the cluster, for example history of website visits, financial transactions, online......\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Use Apache Kafka with PHP - Cloudkul Use Apache Kafka with PHP\" \/>\n<meta property=\"og:description\" content=\"By the help of Kafka we can transfer continuously streaming data to the cluster, for example history of website visits, financial transactions, online......\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/\" \/>\n<meta property=\"og:site_name\" content=\"Cloudkul\" \/>\n<meta property=\"article:published_time\" content=\"2020-04-17T12:33:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-04-17T13:10:21+00:00\" \/>\n<meta name=\"author\" content=\"Siddharth Chouradiya\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/\",\"url\":\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/\",\"name\":\"Use Apache Kafka with PHP - Cloudkul Use Apache Kafka with PHP\",\"isPartOf\":{\"@id\":\"https:\/\/cloudkul.com\/blog\/#website\"},\"datePublished\":\"2020-04-17T12:33:26+00:00\",\"dateModified\":\"2020-04-17T13:10:21+00:00\",\"author\":{\"@id\":\"https:\/\/cloudkul.com\/blog\/#\/schema\/person\/9303394e1ef74da24b7bb4ac81e88706\"},\"description\":\"By the help of Kafka we can transfer continuously streaming data to the cluster, for example history of website visits, financial transactions, online......\",\"breadcrumb\":{\"@id\":\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/cloudkul.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Use Apache Kafka with PHP\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/cloudkul.com\/blog\/#website\",\"url\":\"https:\/\/cloudkul.com\/blog\/\",\"name\":\"Cloudkul\",\"description\":\"Host your eCommerce Store on AWS with Optimized Performance\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/cloudkul.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/cloudkul.com\/blog\/#\/schema\/person\/9303394e1ef74da24b7bb4ac81e88706\",\"name\":\"Siddharth Chouradiya\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/cloudkul.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/73437ba66ba4fee2da26aebcba96530e?s=96&d=https%3A%2F%2Fs.gravatar.com%2Favatar%2F6148c37469011bc2f8e491ca8f5de495%3Fs%3D80&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/73437ba66ba4fee2da26aebcba96530e?s=96&d=https%3A%2F%2Fs.gravatar.com%2Favatar%2F6148c37469011bc2f8e491ca8f5de495%3Fs%3D80&r=g\",\"caption\":\"Siddharth Chouradiya\"},\"url\":\"https:\/\/cloudkul.com\/blog\/author\/siddharthchouradiya-cloud360\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Use Apache Kafka with PHP - Cloudkul Use Apache Kafka with PHP","description":"By the help of Kafka we can transfer continuously streaming data to the cluster, for example history of website visits, financial transactions, online......","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/","og_locale":"en_US","og_type":"article","og_title":"Use Apache Kafka with PHP - Cloudkul Use Apache Kafka with PHP","og_description":"By the help of Kafka we can transfer continuously streaming data to the cluster, for example history of website visits, financial transactions, online......","og_url":"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/","og_site_name":"Cloudkul","article_published_time":"2020-04-17T12:33:26+00:00","article_modified_time":"2020-04-17T13:10:21+00:00","author":"Siddharth Chouradiya","twitter_card":"summary_large_image","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/","url":"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/","name":"Use Apache Kafka with PHP - Cloudkul Use Apache Kafka with PHP","isPartOf":{"@id":"https:\/\/cloudkul.com\/blog\/#website"},"datePublished":"2020-04-17T12:33:26+00:00","dateModified":"2020-04-17T13:10:21+00:00","author":{"@id":"https:\/\/cloudkul.com\/blog\/#\/schema\/person\/9303394e1ef74da24b7bb4ac81e88706"},"description":"By the help of Kafka we can transfer continuously streaming data to the cluster, for example history of website visits, financial transactions, online......","breadcrumb":{"@id":"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/cloudkul.com\/blog\/use-apache-kafka-with-php\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/cloudkul.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Use Apache Kafka with PHP"}]},{"@type":"WebSite","@id":"https:\/\/cloudkul.com\/blog\/#website","url":"https:\/\/cloudkul.com\/blog\/","name":"Cloudkul","description":"Host your eCommerce Store on AWS with Optimized Performance","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cloudkul.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/cloudkul.com\/blog\/#\/schema\/person\/9303394e1ef74da24b7bb4ac81e88706","name":"Siddharth Chouradiya","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cloudkul.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/73437ba66ba4fee2da26aebcba96530e?s=96&d=https%3A%2F%2Fs.gravatar.com%2Favatar%2F6148c37469011bc2f8e491ca8f5de495%3Fs%3D80&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/73437ba66ba4fee2da26aebcba96530e?s=96&d=https%3A%2F%2Fs.gravatar.com%2Favatar%2F6148c37469011bc2f8e491ca8f5de495%3Fs%3D80&r=g","caption":"Siddharth Chouradiya"},"url":"https:\/\/cloudkul.com\/blog\/author\/siddharthchouradiya-cloud360\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/posts\/6938"}],"collection":[{"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/users\/33"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/comments?post=6938"}],"version-history":[{"count":29,"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/posts\/6938\/revisions"}],"predecessor-version":[{"id":7480,"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/posts\/6938\/revisions\/7480"}],"wp:attachment":[{"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/media?parent=6938"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/categories?post=6938"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudkul.com\/blog\/wp-json\/wp\/v2\/tags?post=6938"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}