LAB4_Streaming
_ ==================>
_
-
Select
Sinks > Add
in the menu on the left side -
In the
SINKS
box selectcassandra-enhanced
, you might notice than Pulsar already manage a lot of different technologies. Today we will use the SINK implemented by Datastax. -
In field
cloud.secureConnectBundle
upload the secureBundle zip file you downloaded on previous lab. -
In field
auth.username
provide the astra DB user nameastraUser
-
In field
auth.password
provide the astra DB user nameastraPassword1
This is what the user interface looks like for those first fields.
Now scroll to the bottom of the page to keep entering the following settings:
-
In field
topics
provides the topic name we created in LAB2astra
-
Set field
verbose
totrue
, it is useful to show the log on the user interface. -
Click on the banner
click for extra settings
(yes this is a button) we will form inside the create keys. -
Create key
topic.astra.pulsar.products.mapping
with the valueid=value.id,description=value.value
and then clickADD KEY
.
βΉοΈ Information
---------------
This key describes the mapping between Cassandra Table and pulsar Topic.
- Key NAME follows pattern `<topic>.<topic_name>.<cassandra_keyspace_name>.<cassandra_table_name>.mapping`.
- Key VALUE follows pattern `<colum_name>=value.<json_field_path>, ...`
- Create key
ignoreErrors
with the valueALL
and then clickADD KEY
. If a message does not match with the expecting schema/mapping the subscription Pulsar => Cassandra will loop on this message by default (idea is to not loose messages). Here in dev we can afford raising errors.
-
Below the custom keys table locate the check box
Auto Ack
ans disable it. -
In the **Input Topics (at least one) section change default Topic value from cassandra-enhancedSink-input to
astra
. You do not need to change the namespace, simply clickADD TOPIC
.
- You can finally click
ADD
in the very bottom to create your sink.
This is a lot of steps ! This is what look like the form when you completed steps from 6
to 13
- It will take a few seconds for the sink to start. You will first get a message
Sink successfully created
but running instances are still 0.
- This is what it looks like when the sink has started. Instance is
1
and running is also1
- Now select tab
Instances
to see the details and number of messages that are processed in real time.100
in the following screenshot
Alright ! This is now the moment of truth. Back to the CQL Console in Astra DB execute again the following statement:
select * from pulsar.products;
ππ Et VOILA you now have messages coming into the table, congratulation ! ππ This is the end of hands-on for today, hope you enjoyed we have more resources to share with you in the last section, see you there.
π Getting Started | π¨ Kesque.com βοΈ | π’οΈAstra | π Streaming | π οΈ Resources