Apache Kafka
في هذه الصفحة 8 موضوعات
نص الدروس والتمارين باللغة الإنجليزية، بينما تبقى واجهة الموقع وعناوين الدروس بالعربية.
نظرة عامة
الموضوعات
- الوسطاء (Brokers)
- المواضيع (Topics)
- الأقسام (Partitions)
- مجموعات المستهلكين (Consumer Groups)
- الإزاحات (Offsets)
- الترتيب
تعلّم أولًا
4 ساعة تدريب (تقدير)
مخرجات التعلم
- Explain brokers, topics, partitions, replicas, and the log model.
- Choose partition keys to preserve the ordering that matters.
- Explain consumer groups, offsets, commits, and rebalancing.
- Produce and consume with Spring for Apache Kafka, with error handling and a DLT.
- Configure producer durability (
acks, idempotent producer).
الدروس
- 01Kafka كسجلّ موزّع
- 02المواضيع والأقسام والنسخ المتماثلة
- 03المفاتيح والترتيب
- 04مجموعات المستهلكين والإزاحات
- 05Spring for Apache Kafka
- 06معالجة الأخطاء ومواضيع الرسائل الميتة
- 07إعدادات متانة المنتِج
- 08عرض عملي: أحداث الطلبات من البداية إلى النهاية
- 09عرض عملي: ماذا تفعل إعادة الموازنة (rebalance) بالرسائل قيد المعالجة
المختبر والتمارين
Lab
MID-028-LAB1 · Order events on Kafka
Outcomes: 1–5 · Time: ~4 h
- Create
orders(3 partitions).orderspublishesOrderPlacedkeyed by order ID withacks=alland idempotence enabled. inventoryandpaymentsconsume in separate groups with aDefaultErrorHandler, back-off, and a DLT.- Make handlers idempotent by
eventId. - Test: events of one order stay in order; restart a consumer mid-stream and confirm nothing is lost; send a poison record and find it in the DLT.
Acceptance: per-order ordering preserved; consumer restart loses nothing.
Exercises (assessment: partitioning and ordering scenarios; six exercises)
| ID | Outcome | Task |
|---|---|---|
| MID-028-EX01 | 1 | Draw a topic with 4 partitions and replication factor 3 over 3 brokers; mark leaders and followers. |
| MID-028-EX02 | 2 | Pick a key for: order events, per-customer notifications, sensor readings per device. Say what ordering you get. |
| MID-028-EX03 | 3 | A group has 3 consumers and a topic has 2 partitions. Describe the assignment and what happens when one consumer leaves. |
| MID-028-EX04 | 3 | Explain the failure when offsets are committed before processing and the failure when committed after. |
| MID-028-EX05 | 4 | Configure a Spring Kafka error handler that retries 3 times, 1 s apart, then publishes to a DLT. |
| MID-028-EX06 | 5 | Choose acks, min.insync.replicas, and idempotence for payment events and explain the trade-off. |
أظهر الحلول
- EX01 Each partition has one leader and two followers on different brokers; leaders spread across brokers.
- EX02 Order events:
orderId(per-order order). Notifications:customerId. Sensors:deviceId. No ordering across keys. - EX03 Two consumers get one partition each; one idles. If an active one leaves, rebalance gives its partition to the idle one.
- EX04 Commit before: crash loses the record. Commit after: crash causes redelivery, so handlers must be idempotent.
- EX05
new DefaultErrorHandler(new DeadLetterPublishingRecoverer(template), new FixedBackOff(1000L, 3)). - EX06
acks=all, replication factor 3,min.insync.replicas=2,enable.idempotence=true; cost is extra latency and reduced availability if two replicas are down.
الاختبار
Outcomes: O1 log model · O2 keys and ordering · O3 groups and offsets · O4 Spring and error handling · O5 producer durability.
اختبار ذاتي لك: لا توجد درجات، ولا يُحفظ أو يُرسل أي شيء. اختر إجابة ثم تحقق منها، ويمكنك المحاولة مرة أخرى كما تشاء.
الجزء العملي
Six exercises MID-028-EX01 to EX06 plus the lab. Pass: at least five exercises correct, per-order ordering demonstrated, no loss on consumer restart, and a poison record found in the DLT.
أخطاء شائعة
Random keys where order matters; auto-commit before processing; one partition for everything.
التقييم
Partitioning and ordering scenarios.
الصلة بالمسار الوظيفي
Large-scale messaging (SRB-006).